5 posts tagged with "search"

Beyond Keyword Matching​

Imagine searching for a document using the phrase: "How can I improve application performance?"

You know the information exists, but the search system returns little or nothing useful because the document contains terms such as "latency optimization," "resource utilization," or "performance tuning" instead of the exact words used in the query.

The information is relevant. The wording is simply different.

This is one of the biggest limitations of traditional search systems, especially when working with large volumes of technical or domain-specific data.

The Challenge: Building a Search Engine​

A few weeks into my internship, I was handed an exciting but daunting project: building a fast, responsive search engine that could handle rapid keystrokes and deliver instant search hints.

Inheriting a Black Box​

A month into my internship, I was assigned to work on a project involving the identification of Ayurvedic plants from images using a Convolutional Neural Network (CNN).

For readers unfamiliar with the term, a CNN is a type of deep learning model commonly used for image recognition tasks because it learns visual patterns such as edges, textures, and shapes directly from images.

The project was already underway before I joined, so I inherited work that had been developed by previous interns and students. Along with the project, I received a trained CNN model and the dataset it had supposedly been trained on.

At first glance, that sounded sufficient.

It wasn't.

There was no training code. No preprocessing pipeline. No documentation explaining how predictions mapped to plant names. Just a trained model file that was approximately 6.92 GB in size.

Large Language Models (LLMs) are increasingly being used for structured information extraction from documents such as resumes, invoices, and reports. However, different LLMs behave differently in terms of extraction accuracy, execution time, consistency, and output quality. Choosing the right model for document extraction tasks therefore becomes an important challenge.

To address this, we built an LLM Document Extraction Benchmark System that compares multiple LLMs on structured document extraction tasks. The framework evaluates models using common prompts and documents, then measures their performance using metrics such as execution time, accuracy, precision, recall, and F1 score.

Imagine a company handling thousands of resumes. Now imagine trying to search for the right information using only exact keywords. Traditional search systems often fail because they mainly rely on keyword matching instead of understanding meaning.

This is where vector databases shine with semantic search.