From Library Stacks to Semantic Search: Finding Great Non‑Fiction

The quiet magic of a good shelf

Picture your first week of college: you’re not enrolled in anything glamorous, you’re shelver-shuffling. The job is repetitive in the way only library work can be—read a label, place a book, repeat. At some point you start doing what bored humans always do: you sneak curiosity into the margins.

For me (and for a lot of us who grow into researchers), that meant flipping to a random page and stealing one sentence. Most days it was mildly interesting. Sometimes it detonated into fascination. And the best part wasn’t even the one book—it was the accidental chain reaction. If one book near you was worth reading, the neighbors often were too.

That’s the real lesson hidden inside the most unglamorous workflow imaginable: good discovery is curated randomness. A classification system and an academic library’s selection already bias the world toward “books that survived the attention economy.”

So what happens when the shelves go away and discovery turns into search boxes that rank the loudest pages instead of the best books? The problem isn’t “the internet.” The problem is the type of matching.

This is where semantic search shows up—especially when the goal isn’t to generate answers, but to find reading that will outlast the current feed.

Why “semantic search” feels different

Traditional search is word-based. You type “modern France,” and the engine hunts for pages containing those exact terms (or close variants). That’s not wrong; it’s fast and predictable.

But book discovery is often not that clean. You don’t always remember the phrase the author used. Sometimes you remember the shape of what you want: the vibe of a topic, the kind of argument, the angle of a history, the tone of a biography.

Semantic search tries to match meaning rather than spelling. The mechanism under the hood is usually the same: it represents text as vectors—arrays of numbers—in a way that makes “similar meanings” land near each other in that number space.

Those vectors come from something called an embedding model.

• An embedding is a numeric representation of text (a title, a paragraph, sometimes an entire document).
• A vector is just the embedding written as a list of numbers.
• A vector search or vector database is an index that can retrieve vectors that are “closest” according to a similarity measure.

In practice, closeness is often computed with cosine similarity, a metric that compares the angle between two vectors (not their raw length). Cosine similarity is commonly used because it behaves well for high-dimensional embeddings. (en.wikipedia.org)

So the search pipeline becomes:

1. Convert every book record (title + metadata + maybe descriptions) into an embedding vector.
2. When you search for “social history,” convert the query text into an embedding vector too.
3. Find the nearest neighbors in the vector index—books whose vectors are most similar to the query vector.

That’s semantic search in one breath. The reason it feels magical is simple: you can search with language that doesn’t exactly overlap with the stored metadata, and the math still brings relevant items to the top.

The “AI slop” contrast: retrieval beats generation

The phrase “AI slop” is doing rhetorical work, but the underlying distinction is technical.

• Generation systems (chatbots, summarizers) produce new text. They’re great at fluency, but they can be great at inventing too.
• Retrieval systems fetch existing text or metadata. Their output is constrained by what’s already in the index.

A book-discovery tool that uses semantic search is a retrieval problem first. The model’s main job is not to write a convincing passage about Edith Wharton or French policy—it’s to map your query into the same meaning-space as the books.

That means the quality bar can be set upstream: pick candidates from prize lists, shortlist pages, or other curated sources. The semantic layer then helps you navigate that curated set with less brittle matching.

Building a “Book Prize Index” style system

The most interesting engineering here is boring on purpose: get the dataset in shape, then make the retrieval feel like browsing.

A practical architecture looks like this:

1) Curate the candidate pool

If the tool starts with prize winners and finalists for non-fiction, it inherits an important property: the dataset is already biased toward books that have earned sustained attention.

Quality is still messy—prizes are imperfect proxies—but for discovery, it’s a strong starting filter.

2) Create searchable metadata

Each book record typically gets:

• title
• author
• prize name
• year (and whether winner/finalist)
• optional tags or topics (if available)
• optional short description text

The more consistent and text-rich the record, the more useful the embeddings become.

3) Compute embeddings once

Embedding every record on every query would be wasteful. So you compute embeddings offline (or as a build step), then store them.

4) Store embeddings in a vector index

A vector database or vector index is a specialized structure that stores embeddings and supports fast nearest-neighbor search. (en.wikipedia.org)

You don’t need to understand all of its internals to benefit from it. The user-level effect is that semantic search becomes fast enough to feel interactive.

5) Query-time ranking and filters

Now retrieval can be “browsable.” You can rank by semantic similarity, then apply filters like:

• prize
• year
• region or subject category (if your metadata has it)

That’s how you get the library-shelf vibe: curated candidates, then a meaningful similarity sort.

Where Claude Code fits: automating the unglamorous parts

A system like this lives or dies on data plumbing: collecting lists of finalists, normalizing titles, deduplicating records, and wiring the pipeline.

This is where Claude Code comes in.

Claude Code is an Anthropic command-line agent intended to operate on a code repository: it can read files, edit code, run commands, and (with safety checks) help automate development workflows. (support.claude.com)

The nice part is not that it “does the thinking.” The nice part is that it reduces the time spent on repetitive glue work—turning messy sources into consistent records and keeping the retrieval pipeline coherent.

A tiny end-to-end demo (conceptual)

Here’s the core logic, shown without committing to a specific vendor.

# Pseudocode: semantic search over precomputed embeddings

# 1) Precompute embeddings for each book record
book_vectors = [
  {
    "id": book_id,
    "text": f"{title} {topics} {prize_name}",
    "vector": embed(f"{title} {topics} {prize_name}")
  }
  for book_id, title, topics, prize_name in dataset
]

# 2) At query time, embed the user query
q_vec = embed(user_query)

# 3) Rank by cosine similarity
results = sorted(
  book_vectors,
  key=lambda b: cosine_similarity(q_vec, b["vector"]),
  reverse=True
)

# 4) Return top N
return results[:20]

Once you’ve got that working, the “weird choices” start to make sense. Semantic search will surface books that match meaning, not literal phrases. If your library-browsing brain wants “classic biographies that are surprisingly strange,” the vector space can plausibly reward that kind of abstraction.

Hosting it without losing the plot

One more subtle point: a tool that’s “free” isn’t automatically cheap. Semantic search still costs money when you scale the number of queries, especially if embedding calls or external inference APIs are involved.

On the infrastructure side, platforms like Vercel offer a hobby/free tier and paid plans. (vercel.com)

So the engineering trick is to keep inference costs low at runtime—compute embeddings offline, cache results, and treat the model as a mapping tool rather than a constant generator.

The result is a system that behaves more like an index card catalog than a talkative autocomplete.

The bigger takeaway: build retrieval that respects reality

This is the quiet counter-move against AI slop. Instead of asking models to invent, build tools that guide readers toward existing work.

When discovery is semantic, it can still be grounded:

• curated candidates (prize lists, finalists)
• embeddings that map queries by meaning
• vector search that retrieves neighbors in that meaning space

Why does it work so well for non-fiction? Because non-fiction is where readers actually live in nuance—tone, argument structure, historical context. Semantic search is the first kind of search that can feel like browsing rather than keyword policing.

The bookshelf was a filter plus randomness. A semantic book index can be a filter plus meaning-aware retrieval. Different interface, same impulse: to spend attention on books that were already worth the attention in the first place.

References (technical)

• Claude Code overview and use cases (Anthropic Help Center). (support.claude.com)
• Vector search and vector database concepts. (mongodb.com)
• Cosine similarity definition and its use in similarity search. (en.wikipedia.org)
• Vercel pricing and hobby/free plan documentation. (vercel.com)