On Mon, May 20, 2019 at 6:33 AM Amirouche Boubekki <xxxxxx@gmail.com> wrote:
I put the search engine with partial index ie. I did not crawl everything listed in https://git.sr.ht/~amz3/guile-gotofish/tree/master/roots.txt

The search engine is at http://search.scheme-lang.com/

Please try it and let me know if you see any bug.

Very nice.  I checked for terms from a few pages of srfi.schemers.org and small.r7rs.org.  I got some results, but not all of the ones I expected, but you did say that the index was incomplete.  (For example, I didn't get any hits from the page PetrofskyCatastrophe from small.r7rs.org.)

I have a few suggestions:
  1. Include the search box at the top of results pages, too.  That way, the user doesn't have to go back to the home page to initiate another search.
  2. Ranking needs work.  For searches that get a lot of results, the ordering of results matters a lot.  For example, try searching for "srfi".  There are many results, but the SRFI home page isn't among them even though that page is a more relevant result than any of the ones returned.  It might be enough to apply an algorithm like Google's original PageRank, using as input only the links on the pages in your index.  Given this relatively small data set, it shouldn't be too hard to implement.
Thanks for doing this.  It's exciting to see a Scheme implementation of search that we can actually use for practical purposes!