Implementing Semantic Search: Beyond Keyword Matching

The Limits of Keyword Search

Traditional search matches keywords. If a user searches for "laptop" they won't find results about "notebook computers." Semantic search understands that these terms have the same meaning, dramatically improving search quality.

Vector Embeddings Explained

Embeddings convert text into numerical vectors where similar meanings cluster together in vector space. "King" and "queen" will have similar vectors because they share semantic relationships.

Implementation with Laravel and pgvector

// Migration
Schema::create('searchable_content', function (Blueprint $table) {
    $table->id();
    $table->morphs('searchable');
    $table->text('content');
    $table->vector('embedding', 1536);
    $table->timestamps();
});

// Model
class SearchableContent extends Model
{
    use HasVector;

    protected $vectorColumn = 'embedding';
}

Indexing Content

class SearchIndexer
{
    public function index(Model $model): void
    {
        $content = $this->extractContent($model);
        $embedding = $this->embeddings->generate($content);

        SearchableContent::updateOrCreate(
            [
                'searchable_type' => get_class($model),
                'searchable_id' => $model->id,
            ],
            [
                'content' => $content,
                'embedding' => $embedding,
            ]
        );
    }

    private function extractContent(Model $model): string
    {
        return match (get_class($model)) {
            Product::class => "{$model->name} {$model->description} {$model->category}",
            Article::class => "{$model->title} {$model->body}",
            default => (string) $model,
        };
    }
}

Search Implementation

class SemanticSearch
{
    public function search(string $query, array $options = []): Collection
    {
        $queryEmbedding = $this->embeddings->generate($query);

        return SearchableContent::query()
            ->selectRaw('*, embedding <=> ? as distance', [$queryEmbedding])
            ->when($options['type'] ?? null, fn ($q, $type) =>
                $q->where('searchable_type', $type)
            )
            ->orderBy('distance')
            ->limit($options['limit'] ?? 20)
            ->get()
            ->map(fn ($result) => [
                'item' => $result->searchable,
                'score' => 1 - $result->distance,
                'content' => $result->content,
            ]);
    }
}

Hybrid Search

class HybridSearch
{
    public function search(string $query): Collection
    {
        $semantic = $this->semanticSearch($query);
        $keyword = $this->keywordSearch($query);

        // Reciprocal Rank Fusion
        $scores = [];
        $k = 60; // Fusion constant

        foreach ($semantic as $rank => $result) {
            $id = $result['item']->id;
            $scores[$id] = ($scores[$id] ?? 0) + 1 / ($k + $rank);
        }

        foreach ($keyword as $rank => $result) {
            $id = $result->id;
            $scores[$id] = ($scores[$id] ?? 0) + 1 / ($k + $rank);
        }

        arsort($scores);
        return $this->fetchByIds(array_keys($scores));
    }
}

Conclusion

Semantic search dramatically improves search quality by understanding intent. Combine it with keyword search for best results, and always measure search quality with real user queries.