Docsity
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Docsity AI
Docsity AI
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Mini Search Engine - Project Report | CSCE 3110, Study Guides, Projects, Research of Computer Science

University of North Texas (UNT)Computer Science

Material Type: Project; Class: Data Structures and Algorithms; Subject: Computer Science/Computer Engineering; University: University of North Texas; Term: Spring 2006;

Typology: Study Guides, Projects, Research

Pre 2010

Uploaded on 08/19/2009

koofers-user-npl
koofers-user-npl ๐Ÿ‡บ๐Ÿ‡ธ

10 documents

1 / 3

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
MINI SEARCH ENGINE
CSCE 3110 Data Structures Project
Total: 200 points
Due date: Pre-finals week
Presentation: 05/02-04/2006
Program: 05/04/2006
Report: 05/04/2006
Introduction
In this project, you will be designing and implementing a mini search engine. You are
probably familiar with Google, Altavista or Yahoo, which are some of the most popular
search engines that you can find on the Web. The task performed by a search engine is, as
the name says, to search through a collection of documents. Given a set of texts and a
query, the search engine will locate all documents that contain the keywords in the query.
The problem may be therefore reduced to a search problem, which can be efficiently
solved with the data structures we have studied in this class.
Your task
Your task is to design and implement an algorithm that searches a collection of
documents. You will be provided with a set of 50 documents and a set of sample queries.
You have the freedom to select the data structures and algorithms that you consider to be
more efficient for this task. Of course, you will have to justify your decisions.
First, you will process the documents and store their content (i.e. words / tokens) in the
data structures that you selected (in information retrieval, this phase is called indexing).
Next, for every input query, you will process the query and search its keywords in the
documents, using the previously implemented data structures and an algorithm of your
choice. (this phase is called retrieval). For each such query, you will have to display the
documents that satisfy the query.
The queries may contain simple Boolean operators, that is AND and OR, which act in a
similar manner with the well known analogous logical operators. For instance, a query:
โ€œKeyword1 AND Keyword2โ€ should retrieve all documents that contain both these
keywords (elements). โ€œKeyword 1 OR Keyword 2โ€ instead will retrieve documents that
contain either one of the two keywords.
Example
Consider the following sample documents.
Doc1: I like the class on data structures and algorithms.
Doc2: I hate the class on data structures and algorithms.
Doc3: Interesting statistical data may result from this survey.
pf3

Partial preview of the text

Download Mini Search Engine - Project Report | CSCE 3110 and more Study Guides, Projects, Research Computer Science in PDF only on Docsity!

MINI SEARCH ENGINE

CSCE 3110 Data Structures Project

Total: 200 points

Due date: Pre-finals week

Presentation: 05/02-04/

Program: 05/04/

Report: 05/04/

Introduction In this project, you will be designing and implementing a mini search engine. You are probably familiar with Google, Altavista or Yahoo, which are some of the most popular search engines that you can find on the Web. The task performed by a search engine is, as the name says, to search through a collection of documents. Given a set of texts and a query, the search engine will locate all documents that contain the keywords in the query. The problem may be therefore reduced to a search problem, which can be efficiently solved with the data structures we have studied in this class.

Your task Your task is to design and implement an algorithm that searches a collection of documents. You will be provided with a set of 50 documents and a set of sample queries. You have the freedom to select the data structures and algorithms that you consider to be more efficient for this task. Of course, you will have to justify your decisions.

First, you will process the documents and store their content (i.e. words / tokens) in the data structures that you selected (in information retrieval, this phase is called indexing ). Next, for every input query, you will process the query and search its keywords in the documents, using the previously implemented data structures and an algorithm of your choice. (this phase is called retrieval ). For each such query, you will have to display the documents that satisfy the query.

The queries may contain simple Boolean operators, that is AND and OR, which act in a similar manner with the well known analogous logical operators. For instance, a query: โ€œ Keyword1 AND Keyword2 โ€ should retrieve all documents that contain both these keywords (elements). โ€œ Keyword 1 OR Keyword 2 โ€ instead will retrieve documents that contain either one of the two keywords.

Example Consider the following sample documents. Doc1: I like the class on data structures and algorithms. Doc2: I hate the class on data structures and algorithms. Doc3: Interesting statistical data may result from this survey.

Here are the answers to some queries: Query 1: data Doc1, Doc2, Doc Query2: data AND structures Doc1, Doc Query 3: like OR survey Doc1, Doc

Hints Take a look first at the format of the documents. (They will be available from the class webpage). You will have to parse the input. You may ignore all lines starting with โ€œ<โ€, these are all SGML tags that are useful for certain tasks, but you will probably not find them very useful in this project. The punctuation is already separated from the words, so you do not have to worry about that. You will have to read one word at a time and add it to your data structure. As data structures, you may consider using dictionaries / hashtables, trees โ€“ such as 2- trees, AVL trees, balanced binary search trees. Any other data structures are admissible as well โ€“ although, again, you have to be able to justify your selection. For every word, you should store a list of documents where it occurs, in order to allow for efficient searches and Boolean operators later on.

What to turn in

There are three main parts in this project, all of them contributing to the final project grade.

  1. You will have to write a project report (about 7-10 typed pages โ€“ single space โ€“ 12pt font) that includes:
    • design issues, what are the problems and how you solve them
    • data structures you use, the decision behind selecting them
    • algorithms you employ, again with a justification of your decision
    • particular emphasis should be placed on the running time of your algorithm
    • optimization issues: what could you do to further optimize your algorithm
    • you need to specifically address the problem of scalability: would you implementation be efficient in the case of very large text collections?
    • the report will also include results you obtained for the given sample queries, and for at least five additional queries of your choice
    • any other remarks about your design and implementation The due date for your final written report is 05/04/2006 in class.
  2. You will have to send in a fully working program, written in C/C++, that you can provide with a query, and obtain the list of documents. It is mandatory that you include a README file, as detailed as possible, including compilation and running instructions. Is it also mandatory that your programs are fully documented, that is they should include detailed comments on what is included in each file and what each method does. Submit your programs using the project command on the CSP machines โ€“ project name is