LLM (Llama 2) Web Crawler Seed Expander

Tsubasa Kato
Sep 30, 2023

--

This is a concept demo of using LLM (Llama 2 by Meta) to expand a query that is inputted by the LLM and send the web crawler a JSON response that contains the expanded words (in this case sentences that are generated) as seed to start crawling.

©️Tsubasa Kato

Inspire Search Corporation

LinkedIn: https://www.linkedin.com/in/tsubasakato

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

--

--

Tsubasa Kato
Tsubasa Kato

Written by Tsubasa Kato

Japanese Search Engineer. Likes Quantum Computers and handheld computers.

No responses yet

Write a response