Download Understanding Precision and Recall in Information Retrieval Systems and more Study notes Computer Science in PDF only on Docsity!
Performance
Evaluation
performance of an information
retrieval system?
systems?
Cleverdon – The
Cranfield Experiments
1950s/1960s
- Time Lag. The interval between the demand being made and the answer being given.
- Presentation. The physical form of the output.
- User effort. The effort, intellectual or physical, demanded of the user.
- Recall. The ability of the system to present all relevant documents.
- Precision. The ability of the system to withhold non-relevant documents.
Cleverdon says
Precision & Recall
- Measure ability to find the
relevant information
- If a system can’t identify
relevant information, what use
is it?
Why not the others?
- According to Cleverdon:
- Time lag
- Presentation
- successful if the user can read and understand the list of references returned
- User effort
- can be measured with a straightforward examination of a small number of cases.
In Reality
- Need to consider the user task
carefully
- Cleverdon was focusing on
batch interfaces
- Interactive browsing interfaces
very significant (Turpin & Hersh)
- Interactive systems
- User effort & presentation very
important
In Spite of That
- Precision & Recall
- Usability
Why Not Usability
- Usability requires a user-study
- Every new feature needs a new study (expensive)
- High variance – many confounding factors
- Offline analysis of accuracy
- Once a dataset is found
- Easy to control factors
- Repeatable
- Automatic
- Free
- If the system isn’t accurate, it isn’t going to be usable
Precision & Recall
- For a query, documents are
relevant or not
- The Relevant Set
- The Non-Relevant Set
- A system retrieves a set of
documents
- The Selected Set
- The Non-selected Set
- Relevance is binary!
- Either a document is valuable or not, nothing in between
Precision and Recall
Selected Not Selected Total
Relevant Nrs N rn N r
Irrelevant N is N in Ni
Total N s N n N
Precision
- Percentage of documents selected
that are relevant
- Probability that a selected
document is relevant
- How well does the system filter out
non-relevant stuff
s
rs
N
N
P =
Recall
- Percentage of all relevant
documents selected
- Probability that a given relevant
document will be retrieved
- How complete are the results?
r
r s
N
N R =
How Big is the Selected
Set?
Precision vs. Recall
- Which one is more important?
- Depends on the task!
- Generic web search engine
- Index of court cases
- Need all legal precedents!
- Recall!
Interpolation in
Precision/Recall
Measurement
- Precision/Recall Table or Graph
- Computing average precision
- At fixed levels of recall (0,10,...,90,100)
- Across many different queries
- Each query has a different number of relevant documents - Some queries – can’t compute precision at the fixed level of recall - Imagine 5 relevant documents to query – can only compute precision at recall of 0, 20, 40, 60, 80, 100.
Interpolation Solution
- If precision is not measured at
exact recall level
- Use precision measured at next highest level of recall
- For 5 relevant document example
- For 10% fixed recall level, use the precision measured at 20%
- Note this means that recall at 10% and recall at 20% will be identical for that query
Precision vs. Recall
Rong Jin, Alex G. Hauptmann, Cheng Xiang Zhai. Title language model for information retrieval. 2002. Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval.
Average precision across all user queries
Reference Data
Collections
- TREC
- Text Retrieval Conference
- CACM
- Communications of the ACM
- ISI
- Institute for Scientific Information
TREC
- Much more than a dataset
- Text Retrieval Conference
- Started in 1992 by Donna Harman
- Sponsored by the National Institute of Standards and Technology (US Commerce Dept.) and the Defense Advanced Research Projects Agency (DARPA)
- Held every year
- Only those who perform the TREC retrieval tasks for that year can participate
A TREC Conference
- Split into tracks
- Each track evaluates a different user task
- 2003 Tracks
- Cross-language
- Filtering
- Genome
- High accuracy
- Interactive
- Novelty
- Question-Answering
- Robust retrieval
- Digital video
- Web
TREC Procedure
(for the ad-hoc task)
- Each participant is provided
- information requests , known as topics
- Document collection
- Each participant must then
- Convert information requests to queries
- Return ranked results
- NIST then
- Identifies the relevant documents (using pooling)
- Reports the precision and recall of each participatingsystem
- Participants
- Write a paper describing their system and analyzing their results
- Present the paper at the TREC workshop
Example Topic
From TREC-7 ad-hoc track Number: 363 transportation tunnel disasters</p> <p><desc> Description:What disasters have occurred in tunnels used for transportation? <narr> Narrative:A relevant document identifies a disaster in a tunnel used for trains, motor vehicles, or people. Wind tunnels and tunnels used for wiring, sewage, water, oil,etc. are not relevant. The cause of the problem may be fire, earthquake, flood, or explosion and can beaccidental or planned. Documents that discuss tunnel disasters occurring during construction of a tunnelare relevant if lives were threatened.</p> <p></top></p> <h2>Documents Used in</h2> <h2>TREC</h2> <ul> <li>Wall Street Journal, Federal</li> </ul> <h3>Register, Associated Press,</h3> <h3>Department of Energy</h3> <h3>abstracts, Financial Times,</h3> <h3>Congressional Record, Foreign</h3> <h3>Broadcast Information Service,</h3> <h3>Los Angeles Times, .GOV web</h3> <h3>documents, references from</h3> <h3>MEDLINE</h3> <p><strong>Obtaining TREC Data</strong></p> <ul> <li>Not Free</li> <li>Must sign data use agreement</li> </ul> <p><strong>CACM</strong></p> <ul> <li>Articles about computer</li> </ul> <h3>science research and practice</h3> <ul> <li>Includes metadata <ul> <li>Authors</li> <li>Dates</li> <li>Subject Categories</li> <li>Bibliographic linkage and co-</li> </ul></li> </ul> <h4>citation</h4> <p><strong>ISI</strong></p> <ul> <li>Documents about information</li> </ul> <h3>science</h3> <ul> <li>Metadata <ul> <li>Bibliographic co-citations</li> </ul></li> </ul> <h2>Bibliographic Citation</h2> <h2>Information in CACM</h2> <ul> <li>Direct references <ul> <li>A referenced B</li> </ul></li> <li>Bibliographic coupling <ul> <li>[a, b, n] (n documents were cited by both a and b)</li> <li>Indicates potential similarity</li> </ul></li> <li>Co-citations <ul> <li>[a, b, n] (a and b appear together in a bibliography in n different documents)</li> </ul></li> </ul> <h2>Many Other Datasets of</h2> <h2>Interest</h2> <ul> <li>Many other datasets are available</li> </ul> <h4>besides the ones mentioned</h4> <ul> <li>Most don’t have relevance data</li> <li>Others</li> <li>EachMovie – collaborative filtering</li> <li>Machine Learning Repository at U. of California – Irvine.</li> <li>Includes a SPAM dataset</li> <li>Most don’t have text</li> </ul> <h2>Query Language</h2> <h2>Features</h2> <ul> <li>Important to understand the</li> </ul> <h3>different query capabilities of</h3> <h3>the popular search engines</h3> <p><strong>Query Elements</strong></p> <ul> <li>Words (index terms) <ul> <li>Generally automatically</li> </ul></li> </ul> <h4>extracted</h4> <ul> <li>What defines a word? <ul> <li>Separated by spaces?</li> <li>What about other characters?</li> <li>Contractions?</li> <li>Hyphens?</li> </ul></li> </ul> <p><strong>Other Query Options</strong></p> <ul> <li>Phrases <ul> <li>“Oh! the difference of man and man” ( <em>K</em> <em>Lear, Act iv, Sc.2</em> )</li> </ul></li> <li>Proximity <ul> <li>“Konstan w/2 Herlocker” (Citeseer)</li> <li>“Destruction <near> Oregon” (NSF – Verity) - <strong>Tsunamis Affecting the United States</strong> <strong>West Coast 1812-</strong></li> </ul></li> </ul> <h2>Consider Indexing of</h2> <h2>Proximity...</h2> <ul> <li>Phrases <ul> <li>Could treat each separate</li> </ul></li> </ul> <h4>phrase as a keyword</h4> <ul> <li>Combinatorial expansion!</li> <li>What about NEAR?</li> </ul> <h2>Other Query</h2> <h2>Possibilities</h2> <ul> <li>Prefixes (also known as truncation) <ul> <li>(Altavista) Herloc* <ul> <li><strong>Welcome to Herlocher's Dipping Mustard!!</strong></li> </ul></li> </ul></li> <li>Other string patterns <ul> <li>Substrings, suffixes, regular expressions, etc</li> </ul></li> <li>Accounting for errors <ul> <li>glimpse -1 unbelievable (allows one character spellingerror)</li> </ul></li> <li>Soundex <ul> <li>“Sounds like”</li> <li>Popular in genealogy and the census</li> </ul></li> <li>Examine the documentation for your favorite search engines</li> <li>A nice site with a feature table is <ul> <li>http://searchengineshowdown.com/features/</li> </ul></li> </ul> <p><strong>Some IR Challenges</strong></p> <ul> <li>Users need information</li> <li>They often can’t describe their need perfectly - i.e. Anomalous State of Knowledge (ASK)</li> <li>Or perhaps they can only describe it in “their” terminology - Which might be different from others</li> <li>Or perhaps they are lazy (or time constrained) - Can’t/won’t spend the effort to design a “good” query</li> </ul> <p><strong>Query Expansion!</strong></p> <ul> <li>Can we automatically expand</li> </ul> <h3>the query?</h3> <ul> <li>Goal: improve the</li> </ul> <h3>effectiveness of search by</h3> <h3>creating a “better” query</h3> <p><strong>Relevance Feedback</strong></p> <ul> <li>User issues short, simple query</li> <li>From retrieved results <ul> <li>User indicates relevant documents</li> </ul></li> <li>Relevant documents are</li> </ul> <h4>considered an “example” of a more</h4> <h4>detailed query</h4> <ul> <li>Find documents similar to the</li> </ul> <h4>identified relevant documents</h4> <h2>Evaluating Relevance</h2> <h2>Feedback</h2> <ul> <li>Need to do a user experiment <ul> <li>Relevant documents selected by</li> </ul></li> </ul> <h4>user depend on the original</h4> <h4>search results</h4> <ul> <li>Precision and Recall <ul> <li>Suppose you measure P&R of</li> </ul></li> </ul> <h4>the results from the expanded</h4> <h4>query?</h4> <h2>Prec. & Recall with</h2> <h2>Relevance Feedback</h2> <ul> <li>User provides initial query</li> <li>System provides initial results (say 10)</li> <li>User identifies two relevant documents at ranks 8 and 10</li> <li>System provides updated results based on R. F.</li> <li>The relevant documents that were originally at ranks 8 and 10 are now at ranks 1 and 2 - An easy cheat!</li> </ul> <h2>Precision and Recall &</h2> <h2>Relevance Feedback</h2> <ul> <li>Residual collection precision</li> </ul> <h3>and recall</h3> <ul> <li>Compute precision and recall</li> </ul> <h4>after removing all documents</h4> <h4>used as feedback.</h4> <ul> <li>Consider the user interaction <ul> <li>Could compare relevance</li> </ul></li> </ul> <h4>feedback to manual query</h4> <h4>reformulation</h4> <h2>Fully Automatic Query</h2> <h2>Expansion</h2> <ul> <li>Relevance feedback requires user</li> </ul> <h4>input</h4> <ul> <li>User provides examples of relevant and non-relevant documents</li> <li>System extracts words from those documents and adds them or removes them from the query</li> <li>What can we do without user input?</li> <li>Statistical thesaurus expansion</li> </ul> <p><strong>Consider</strong></p> <ul> <li>User searches for “cat”</li> <li>Won’t see documents that use the</li> </ul> <h4>word “feline” instead of cat</h4> <ul> <li>Thus terms are not independent</li> <li>A potentially better search</li> <li>“cat OR feline”</li> <li>If you had a thesaurus...</li> <li>Look up each query term</li> <li>Add all the synonyms to the query</li> </ul> <h2>Construct a Thesaurus</h2> <ul> <li>May not have a useful</li> </ul> <h3>thesaurus available</h3> <ul> <li>Research has shown that</li> </ul> <h4>generic thesauri don’t fare well</h4> <ul> <li>Lose too much precision</li> <li>Need a thesaurus specific to</li> </ul> <h3>the topics in your query</h3> <ul> <li>Can you construct one?</li> </ul> <h2>Use Document</h2> <h2>Collection to Construct</h2> <h2>Thesaurus</h2> <ul> <li>Analyze the text in the document store</li> <li>Global <ul> <li>Build a thesaurus from all documents</li> </ul></li> <li>Local <ul> <li>First retrieve documents based on initial query</li> <li>Build a thesaurus from those documents</li> </ul></li> <li>Expand query using thesaurus <ul> <li>Rerun query</li> </ul></li> </ul> <h2>Co-occurrences</h2> <ul> <li>If two words frequently co-</li> </ul> <h3>occur</h3> <ul> <li>They might be related</li> <li>For example</li> <li>Any document that mentions</li> </ul> <h4>“feline” frequently will include</h4> <h4>mention “cat” frequently</h4> <h2>Association Clusters</h2> <h5>j l</h5> <h5>u v</h5> <h2>d D</h2> <h2>u v k j k j</h2> <h2>c f f</h2> <p>Correlation</p> <p>Document</p> <p>All Retrieved Documents</p> <p>Frequency of k (^) u in document j</p> </div></div></div></div><footer id="footer" class="sc-gsnTZi hxUvtc"><div width="100%" display="grid" class="sc-gsnTZi sc-breuTD hQDdiS hAbyOH"><div class="sc-gsnTZi sc-ksZaOG hINUYJ ijDdyb"><p color="muted" class="sc-dkzDqf jtFJUi">University</p><div width="100%" display="grid" class="sc-gsnTZi sc-breuTD jxbiSy ddZtgY"><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/university/us/" color="negative" class="sc-dkzDqf liUIpz">United States of America (USA)</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/university/ph/" color="negative" class="sc-dkzDqf liUIpz">Philippines</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/university/in/" color="negative" class="sc-dkzDqf liUIpz">India</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/university/vn/" color="negative" class="sc-dkzDqf liUIpz">Vietnam</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/university/gb/" color="negative" class="sc-dkzDqf liUIpz">United Kingdom</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/university/ca/" color="negative" class="sc-dkzDqf liUIpz">Canada</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/university/tr/" color="negative" class="sc-dkzDqf liUIpz">Turkey</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/university/id/" color="negative" class="sc-dkzDqf liUIpz">Indonesia</a></div></div></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ ijDdyb"><p color="muted" class="sc-dkzDqf jtFJUi">questions</p><div width="100%" display="grid" class="sc-gsnTZi sc-breuTD jxbiSy ddZtgY"><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/answers/" color="negative" class="sc-dkzDqf liUIpz">Latest questions</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/answers/biology-and-chemistry/" color="negative" class="sc-dkzDqf liUIpz">Biology and Chemistry</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/answers/psicology-and-sociology/" color="negative" class="sc-dkzDqf liUIpz">Psychology and Sociology</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/answers/management/" color="negative" class="sc-dkzDqf liUIpz">Management</a></div><div class="sc-gsnTZi sc-ksZaOG hINUYJ dPsdkt"><a href="/en/answers/physics/" color="negative" class="sc-dkzDqf liUIpz">Physics</a></div></div></div></div><div class="sc-gsnTZi dfEwCf"><img src="https://assets.docsity.com/ds/logo/docsity-logo-rebrand-negativo.svg" alt="Docsity logo" width="269px" height="120px" class="sc-crXcEl kmMsfS"/></div><div direction="row" class="sc-bczRLJ iznJYF"><a color="negative" target="_self" href="/en/ai/explore-ai/" class="sc-dkzDqf liUIpz">Docsity AI</a><a color="negative" target="_self" href="/en/store/sell/" class="sc-dkzDqf liUIpz">Sell documents</a><a color="negative" target="_blank" href="https://corporate.docsity.com/about-us/" class="sc-dkzDqf liUIpz">About us</a><a color="negative" target="_blank" href="https://support.docsity.com/hc/en-us" class="sc-dkzDqf liUIpz">Contact us</a><a color="negative" target="_blank" href="https://corporate.docsity.com/docsity-partners/" class="sc-dkzDqf liUIpz">Partners</a><a color="negative" target="_self" href="/en/pag/how-does-docsity-works/" class="sc-dkzDqf liUIpz">How does Docsity work</a><a color="negative" target="_blank" href="https://www.weuni.com/en/" class="sc-dkzDqf liUIpz">WeUni</a></div><div class="sc-gsnTZi ktkVSn"><div direction="row" class="sc-bczRLJ iznJYF"><a color="negative" target="_self" href="/es/universidad/es/" class="sc-dkzDqf liUIpz">Español</a><a color="negative" target="_self" href="/it/universita/it/" class="sc-dkzDqf liUIpz">Italiano</a><a color="negative" target="_self" href="/en/university/us/" class="sc-dkzDqf liUIpz">English</a><a color="negative" target="_self" href="/sr/univerzitet/sr/" class="sc-dkzDqf liUIpz">Srpski</a><a color="negative" target="_self" href="/pl/uniwersytet/pl/" class="sc-dkzDqf liUIpz">Polski</a><a color="negative" target="_self" href="/ru/universitet/ru/" class="sc-dkzDqf liUIpz">Русский</a><a color="negative" target="_self" href="/pt/universidade/br/" class="sc-dkzDqf liUIpz">Português</a><a color="negative" target="_self" href="/fr/universite/fr/" class="sc-dkzDqf liUIpz">Français</a><a color="negative" target="_self" href="/de/universitat/de/" class="sc-dkzDqf liUIpz">Deutsch</a></div></div><div class="sc-gsnTZi fHZVwh"><div direction="row" class="sc-bczRLJ iznJYF"><a color="negative" target="_self" href="/en/usa/" class="sc-dkzDqf liUIpz">United States of America (USA)</a><a color="negative" target="_self" href="/en/phl/" class="sc-dkzDqf liUIpz">Philippines</a><a color="negative" target="_self" href="/en/ind/" class="sc-dkzDqf liUIpz">India</a><a color="negative" target="_self" href="/en/vnm/" class="sc-dkzDqf liUIpz">Vietnam</a><a color="negative" target="_self" href="/en/gbr/" class="sc-dkzDqf liUIpz">United Kingdom</a><a color="negative" target="_self" href="/en/can/" class="sc-dkzDqf liUIpz">Canada</a><a color="negative" target="_self" href="/en/tur/" class="sc-dkzDqf liUIpz">Turkey</a><a color="negative" target="_self" href="/en/idn/" class="sc-dkzDqf liUIpz">Indonesia</a></div></div><div class="sc-gsnTZi dNwUEv"><div direction="row" class="sc-bczRLJ gIiniM"><button display="flex" aria-label="Google play store" class="sc-gsnTZi gTkYJ"><img src="https://assets.docsity.com/sdk/common/app/google/GooglePlayBadge_EN.svg" alt="Google play badge" width="135px" height="40px"/></button><button display="flex" aria-label="App store" class="sc-gsnTZi gTkYJ"><img src="https://assets.docsity.com/sdk/common/app/apple/AppStoreBadge_EN.svg" alt="App store badge" width="120px" height="40px"/></button></div></div><div display="flex" class="sc-gsnTZi gCHycu"><div direction="row" class="sc-bczRLJ iznJYF"><a color="negative" target="_self" href="/en/pag/terms-and-conditions/" class="sc-dkzDqf liUIpz">Terms of Use</a><a color="negative" target="_self" href="/en/pag/cookie-policy/" class="sc-dkzDqf liUIpz">Cookie Policy</a><button color="negative" target="_self" class="sc-dkzDqf liUIpz">Cookie setup</button><a color="negative" target="_self" href="/en/pag/privacy/" class="sc-dkzDqf liUIpz">Privacy Policy</a></div><div direction="row" class="sc-bczRLJ iznJYF"><a href="https://www.facebook.com/DocsityGlobal" target="_blank" aria-label="facebook" color="inherit" class="sc-dkzDqf kUmfEV"><span class="sc-iIPllB jjxFtg"><svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" fill="currentColor" viewBox="0 0 256 256"><path d="M128,20A108,108,0,1,0,236,128,108.12,108.12,0,0,0,128,20Zm12,191.13V156h20a12,12,0,0,0,0-24H140V112a12,12,0,0,1,12-12h16a12,12,0,0,0,0-24H152a36,36,0,0,0-36,36v20H96a12,12,0,0,0,0,24h20v55.13a84,84,0,1,1,24,0Z"></path></svg></span></a><a href="https://www.instagram.com/docsity_en/" target="_blank" aria-label="instagram" color="inherit" class="sc-dkzDqf kUmfEV"><span class="sc-iIPllB jjxFtg"><svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" fill="currentColor" viewBox="0 0 256 256"><path d="M128,80a48,48,0,1,0,48,48A48.05,48.05,0,0,0,128,80Zm0,72a24,24,0,1,1,24-24A24,24,0,0,1,128,152ZM176,20H80A60.07,60.07,0,0,0,20,80v96a60.07,60.07,0,0,0,60,60h96a60.07,60.07,0,0,0,60-60V80A60.07,60.07,0,0,0,176,20Zm36,156a36,36,0,0,1-36,36H80a36,36,0,0,1-36-36V80A36,36,0,0,1,80,44h96a36,36,0,0,1,36,36ZM196,76a16,16,0,1,1-16-16A16,16,0,0,1,196,76Z"></path></svg></span></a><a href="https://www.linkedin.com/company/docsity-com/" target="_blank" aria-label="linkedin" color="inherit" class="sc-dkzDqf kUmfEV"><span class="sc-iIPllB jjxFtg"><svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" fill="currentColor" viewBox="0 0 256 256"><path d="M216,20H40A20,20,0,0,0,20,40V216a20,20,0,0,0,20,20H216a20,20,0,0,0,20-20V40A20,20,0,0,0,216,20Zm-4,192H44V44H212ZM112,176V120a12,12,0,0,1,21.43-7.41A40,40,0,0,1,192,148v28a12,12,0,0,1-24,0V148a16,16,0,0,0-32,0v28a12,12,0,0,1-24,0ZM96,120v56a12,12,0,0,1-24,0V120a12,12,0,0,1,24,0ZM68,80A16,16,0,1,1,84,96,16,16,0,0,1,68,80Z"></path></svg></span></a></div><div direction="row" class="sc-bczRLJ iznJYF"><a color="negative" target="_self" href="/en/sitemap/better/" class="sc-dkzDqf liUIpz">Sitemap Resources</a><a color="negative" target="_self" href="/en/sitemap/latest/" class="sc-dkzDqf liUIpz">Sitemap Latest Documents</a><a color="negative" target="_self" href="/en/sitemap/country/" class="sc-dkzDqf liUIpz">Sitemap Languages and Countries</a></div><p color="muted" class="sc-dkzDqf exhynh">Copyright © 2026 Ladybird Srl - Via Leonardo da Vinci 16, 10126, Torino, Italy - VAT 10816460017 - All rights reserved</p></div></footer><div id="modal-area"></div><div width="unset,360px" overflow="hidden" class="sc-gsnTZi GrWcS"></div></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"currentUrl":"/en/docs/lecture-slides-on-performance-evaluation-cs-519/6743221/","currentRoute":"document.view","currentUser":null,"translations":{"alerts.action_spacer":"or","alerts.back_lang.action":"Back to %s language","alerts.complete_profile.link":"Complete profile","alerts.complete_profile.text":"Help us recommend the best content for you and receive instantly **%s download points**","alerts.confirm_email.action_modifica":"Change email address","alerts.confirm_email.action_reinvia":"Resend email.","alerts.confirm_email.text":"Confirm your email address by clicking on the link we sent you at %s.","alerts.diff_lang.action":"Click here","alerts.diff_lang.text":"You are browsing in a language other than your own","alerts.invalid_email.action_modifica":"Change email","alerts.invalid_email.action_resend":"Resend email","alerts.invalid_email.text":"We are unable to send the confirmation email to your address: **%s**","alerts.low_points.action":"Get points immediately","alerts.low_points.main":"You are running out of Download Points.","common.date.formatAcademicYear":"Pre %s","completeProfile.form_label":"Birth year","completeProfile.form_label_cta":"Confirm data","completeProfile.label_cta":"Start now","completeProfile.text_default_1":"You'll earn % points","completeProfile.text_default_2":"to download documents and gain access to free Video Courses and Quizzes","completeProfile.text_default_3":"to download documents","completeProfile.text_guest":"To proceed, enter the missing data","completeProfile.title_default":"Update your profile to continue","completeProfile.title_guest":"Oops, looks like you missed something!","docProfessorModal.disclaimer":"The page is generated automatically and is not managed by a person (the Professor or anyone else). The documents listed are not official or representative of the teaching methods and contents of the Professor in question. They are automatically aggregated according to criteria entered by users. The personal data relating to the Professor contained in this page have been collected by the Data Controller (Ladybird s.r.l.) from open sources accessible to the public and / or have been generated directly by the users. The goal is to enrich the contents of the platform for the legitimate interest of Ladybird s.r.l. to improve its usability for the benefit of its users. They will be stored until any opposition to the processing by the interested party, who is entitled to oppose the processing for reasons related to their particular situation, except for the provisions of art. 21 § 1, Paragraph 2, GDPR. For more information on the categories of recipients of the personal data and the additional rights of the data subject, please refer to sections B and H respectively of the Docsity privacy policy available at this [link](%s)","docs.abstractArea.documents":"Documents","docs.abstractArea.seoString":"Download %1$s and more %2$s %3$s in PDF only on Docsity!","docs.abstractArea.thisSubject":"This subject","docs.abstractArea.title":"Partial preview of the text","docs.actionSection.disabledByLanguageUser":"Only %s Docsity users can download this document","docs.actionSection.disabledByUserNotSeller":"We have received reports about this document. The download is temporarily disabled.","docs.actions.addFavourite":"Add to favourites","docs.actions.approveDocument":"Approve Document","docs.actions.approveDocument.force":"Force document approval","docs.actions.cannotEditPrime":"This document can no longer be modified because it has been promoted as a Top Document","docs.actions.deleteDocument":"Delete","docs.actions.download":"Download","docs.actions.downloadDocuemnt":"Download the document","docs.actions.edit":"Edit","docs.actions.follow":"Follow","docs.actions.reconvertDocument.errorMessage":"Error during the request","docs.actions.reconvertDocument.label":"Convert the document","docs.actions.reconvertDocument.successMessage":"Conversion successfully completed","docs.actions.remove":"Remove","docs.actions.removeFavourite":"Remove from favourites","docs.actions.reportDocument":"Report document","docs.actions.requestError":"An error has occurred","docs.actions.requestSuccess":"Request submitted, wait a few seconds and reload the page","docs.actions.review":"Leave a review","docs.actions.save":"Save","docs.actions.shareDocTooltip":"Share document","docs.actions.shareLabel":"Share","docs.actions.unfollow":"Unfollow","docs.actions.useAI":"Try Docsity AI","docs.actions.watchReview":"Your review","docs.aiOffCanvas.aiAlertFreemium":"**Free trial on %s documents or use without limits with Premium**","docs.aiOffCanvas.aiCtaDownload":"Download document","docs.aiOffCanvas.aiDownload":"You can use AI after downloading the document","docs.aiOffCanvas.aiDownloadDocument":"**AI is available with a Premium plan.** You can use AI after downloading the document, no additional cost.","docs.aiOffCanvas.aiFeaturePoints":"%s points","docs.aiOffCanvas.chat.infoPoint1":"Explore the content of a document with chat responses","docs.aiOffCanvas.chat.infoPoint2":"Clear any doubts to study in less time","docs.aiOffCanvas.chat.title":"Ask the document","docs.aiOffCanvas.map.infoPoint1":"Get the conceptual map of a document, generated by the AI","docs.aiOffCanvas.map.infoPoint2":"Edit the map","docs.aiOffCanvas.map.infoPoint3":"Download a PNG","docs.aiOffCanvas.map.title":"Concept map","docs.aiOffCanvas.points":"points","docs.aiOffCanvas.quiz.infoPoint1":"Generate a number of questions on the topics covered by the document","docs.aiOffCanvas.quiz.infoPoint2":"Download the PDF quiz so you can study whenever you want","docs.aiOffCanvas.quiz.infoPoint3":"Get the correct answer for each question","docs.aiOffCanvas.quiz.title":"Quiz","docs.aiOffCanvas.subtitle":"What you can get","docs.aiOffCanvas.summary.infoPoint1":"Summary of approximately 70%, in just a few minutes and in PDF format","docs.aiOffCanvas.summary.infoPoint1updated":"Summarise the document in a few minutes","docs.aiOffCanvas.summary.infoPoint2":"Download the summary document in PDF format","docs.aiOffCanvas.summary.infoPoint2updated":"Download the summary in PDF format","docs.aiOffCanvas.summary.title":"Summary","docs.aiOffCanvas.title":"Use Docsity AI on the document","docs.aiOffCanvas.titleUpdated":"Download the document to use AI","docs.breadcrumb.free":"Documents","docs.breadcrumb.store":"Store","docs.ctaArea.toggleSidebarTooltip":"Find here **related documents** and other useful resources","docs.ctaMessages.AI":"AI","docs.ctaMessages.alreadyDownloadedTitle":"You already downloaded this document","docs.ctaMessages.alreadyReviewedTitle":"You've already downloaded and reviewed this document","docs.ctaMessages.descriptionAI":"Make a summary, create a concept map or a quiz from this document","docs.ctaMessages.disabledByLanguageUser":"Only %s Docsity users can download this document","docs.ctaMessages.disabledByUserNotSeller":"This document is temporarily unavailable for download","docs.ctaMessages.discountBadge":"On special offer","docs.ctaMessages.myDocumentOffer":"Your document is discounted temporarily","docs.ctaMessages.uploadedTtile":"You uploaded this document on %s","docs.docDeletionModal.empty":"The field is mandatory","docs.docDeletionModal.notice":"Document_notice","docs.docDeletionModal.report":"Reports","docs.docDeletionModal.submit":"Delete Document","docs.docDeletionModal.textArea.label":"Notes","docs.docDeletionModal.textArea.optional":"(optional)","docs.docDeletionModal.textArea.placeholder":"Write here...","docs.docDeletionModal.title":"Select the reason for deletion","docs.docMyDiscountModal.close":"OK, close","docs.docMyDiscountModal.content":"Since this document was not receiving downloads, we decided to discount it to. When the number of downloads goes back up, we will increase the price again.","docs.docMyDiscountModal.title":"This document has been discounted","docs.docProfessorModal.disclaimer":"The page is generated automatically and is not managed by a person (the Professor or anyone else). The documents listed are not official or representative of the teaching methods and contents of the Professor in question. They are automatically aggregated according to criteria entered by users. The personal data relating to the Professor contained in this page have been collected by the Data Controller (Ladybird s.r.l.) from open sources accessible to the public and / or have been generated directly by the users. The goal is to enrich the contents of the platform for the legitimate interest of Ladybird s.r.l. to improve its usability for the benefit of its users. They will be stored until any opposition to the processing by the interested party, who is entitled to oppose the processing for reasons related to their particular situation, except for the provisions of art. 21 § 1, Paragraph 2, GDPR. For more information on the categories of recipients of the personal data and the additional rights of the data subject, please refer to sections B and H respectively of the Docsity privacy policy available at this [link](%s)","docs.docReportingModal.email":"Email","docs.docReportingModal.errorMessage":"We received a report from you regarding this document on %1$s with reason *“%2$s“*","docs.docReportingModal.errorTitle":"You have reported this document before","docs.docReportingModal.genericMessage":"We will try to solve this as quickly as possible.","docs.docReportingModal.guestName":"Name and surname","docs.docReportingModal.leaveAmessage":"Leave a message","docs.docReportingModal.optional":"Optional","docs.docReportingModal.reasons.copyrightViolation":"This document contains copyright infringement","docs.docReportingModal.reasons.duplicatedDocument":"This document has been duplicated","docs.docReportingModal.reasons.inconsitentContent":"The content is not consistent with the description","docs.docReportingModal.reasons.uploadedMyDocument":"User has uploaded a document that belongs to me","docs.docReportingModal.required":"Field required","docs.docReportingModal.sendReport":"Send report","docs.docReportingModal.successMessage":"Error processing your request","docs.docReportingModal.successTitle":"Thanks for reporting","docs.docReportingModal.title":"Why do you want to report this document?","docs.docReportingModal.validationError.fieldTooLong":"The max. number of characters is %d","docs.download.idle.buttonLabel":"Download now","docs.download.idle.label":"Download using","docs.download.idle.label2":"It will be available on all devices","docs.download.idle.selectDownload":"%s download Points","docs.download.idle.selectPremium":"%s premium Points","docs.download.idle.title":"You're downloading...","docs.download.noPoints.ctaTitle":"Download now","docs.download.noPoints.powerCtaSubtitle":"by purchasing a Power Recharge","docs.download.noPoints.premiumCtaSubtitle":"by purchasing a Premium plan","docs.download.noPoints.premiumUploadAction":"[Upload documents](%s) and get download points within a **few minutes**","docs.download.noPoints.rewardLabel":"and get the points you are missing","docs.download.noPoints.rewardTimeLabel":"*%s* Points are awarded within a **few minutes**","docs.download.noPoints.title":"Your points:","docs.download.noPoints.uploadAction":"Share documents","docs.download.redirecting.close":"Close","docs.download.redirecting.title":"There was a problem, [try again](%s)","docs.download.resend_email_action":"Resend email","docs.download.resend_email_body":"Before proceeding confirm your email. Click the link that we just sent you at **%s**.","docs.download.resend_email_heading":"Oops! Something is missing to download the document","docs.download.resend_email_invalid_description":"We can't deliver messages to **%s**. Before proceeding, double check whether your email address is valid or you need to change it.","docs.download.resend_email_invalid_title":"Oops! Something is missing to download the document","docs.flatSection.academicYear":"Academic Year","docs.flatSection.description":"Description","docs.flatSection.download":"Download","docs.flatSection.favourites":"Favourites","docs.flatSection.follow":"Follow","docs.flatSection.hide":"Hide","docs.flatSection.multipleDocLabel":"Documents","docs.flatSection.multipleReviewsLabel":"Reviews","docs.flatSection.page":"Page","docs.flatSection.pageNumber":"Number of pages","docs.flatSection.pages":"Pages","docs.flatSection.professorPrefix":"Prof. %s","docs.flatSection.sellDateLabel":"Available from","docs.flatSection.showMore":"Show more","docs.flatSection.singleDocLabel":"Document","docs.flatSection.singleReviewsLabel":"Review","docs.flatSection.unfollow":"Unfollow","docs.flatSection.uploadDateLabel":"Uploaded on","docs.header.alreadyDownloadedTitle":"You already downloaded this document","docs.header.alreadyReviewedTitle":"You've already downloaded and reviewed this document","docs.header.review":"%d Review","docs.header.reviews":"%d Reviews","docs.header.uploadedTtile":"You uploaded this document on %s","docs.headingArea.notPublished":"Document not published","docs.headingArea.topTooltip":"One of the most popular and successful documents in our community","docs.hero.multipleDocLabel":"Documents","docs.hero.notPublished":"Document not published","docs.hero.professorPrefix":"Prof. %s","docs.hero.questionsLabel":"What you will learn","docs.hero.review":"%d Review","docs.hero.reviews":"%d Reviews","docs.hero.sellDateLabel":"Available from","docs.hero.singleDocLabel":"Document","docs.hero.typology":"Typology: %s","docs.hero.unknownUser":"unknown user","docs.hero.uploadDateLabel":"Uploaded on","docs.infoArea.alreadyReviewedTitle":"You already downloaded this document","docs.infoArea.uploadedTtile":"You uploaded this document on %s","docs.intentClose.action":"Show others","docs.intentClose.heading":"Abracadabra 🔮 More documents for you! There’s really no magic, only our massive library!","docs.player.controllerLabel":"Page %1$d / %2$d","docs.player.crossPaywall.buttonPremiumLabel":"Download now","docs.player.crossPaywall.buttonShareLabel":"Share documents","docs.player.crossPaywall.textButtonPremium":"by purchasing a Premium plan","docs.player.crossPaywall.textButtonShare":"and get the points you are missing in **%s hours**","docs.player.downloadBlock.always.buttonLabel":"Download for free","docs.player.downloadBlock.always.copy":"This page cannot be seen from the preview","docs.player.downloadBlock.always.copyLastPage":"You can download it any time from any device","docs.player.downloadBlock.always.title":"You already downloaded this document","docs.player.downloadBlock.always.titleLastPage":"You already downloaded this document","docs.player.downloadBlock.default.buttonLabel":"Download","docs.player.downloadBlock.default.copy":"This page cannot be seen from the preview","docs.player.downloadBlock.default.copyLastPage":"You can download it any time from any device","docs.player.downloadBlock.default.title":"Don't miss anything!","docs.player.downloadBlock.default.titleLastPage":"Download the full document","docs.player.paywallBlock.unlock.buttonLabel":"Read the document","docs.player.paywallBlock.unlock.title":"Register for free to read the full document","docs.player.previewLabel":"Document preview","docs.playerToolbar.pageNumber":"Number of pages","docs.playerToolbar.scrollTop":"Go back to top","docs.priceModal.pointsImgAlt":"Get points","docs.priceModal.pointsLinkLabel":"Other ways to get download points for free","docs.priceModal.pointsSubtitle":"[Upload](%1$s) your documents or [answer](%2$s) questions and get download points","docs.priceModal.pointsTitle":"Get points to download the document","docs.priceModal.premiumImgAlt":"Go Premium","docs.priceModal.premiumLinkLabel":"See our Premium plans","docs.priceModal.premiumSubtitle":"Choose one of our Premium plans and use the points to download your documents right away","docs.priceModal.premiumTitle":"Don't want to wait?","docs.priceSection.discountBadge":"On special offer","docs.priceSection.discountLimitedLabel":"Limited-time offer","docs.priceSection.points":"Points","docs.recentArea.emptyPlaceholder":"Here you'll find the latest visited documents","docs.recentArea.title":"Recently viewed documents","docs.relatedArea.emptyPlaceholder":"There are no similar documents","docs.relatedArea.title":"Related documents","docs.relatedArea.viewOthers":"Show others","docs.reviewsArea.noReviewTitle":"No reviews yet","docs.reviewsArea.title":"Reviews","docs.reviewsArea.viewAll":"View all","docs.reviewsModal.awaitingModeration":"Under moderation","docs.reviewsModal.errorCase":"Something went wrong","docs.reviewsModal.getPoints":"Get %d download points","docs.reviewsModal.leaveReview":"Leave a review","docs.reviewsModal.orderHighest":"Highest rate","docs.reviewsModal.orderLowest":"Lowest rate","docs.reviewsModal.orderRecent":"Most recent","docs.reviewsModal.rating1":"Poor","docs.reviewsModal.rating2":"Insufficient","docs.reviewsModal.rating3":"Sufficient","docs.reviewsModal.rating4":"Good","docs.reviewsModal.rating5":"Excellent","docs.reviewsModal.reload":"Top-up","docs.reviewsModal.showMore":"Show others","docs.reviewsModal.showRatingDetails":"Show more","docs.reviewsModal.sortBy":"Sort by","docs.reviewsModal.title":"Reviews","docs.reviewsModal.whoCanReview":"Only users who downloaded the document can leave a review","docs.reviewsModal.yourReview":"Your review","docs.searchBar.counter":"%(index)s of %(total)s **(%(matches)s visible)**","docs.searchBar.error.button":"Reload","docs.searchBar.error.content":"Try searching again","docs.searchBar.error.title":"Loading error","docs.searchBar.placeholder":"Search in the preview","docs.searchBar.popover.close":"Close","docs.searchBar.popover.download":"Download","docs.searchBar.popover.hasDownload":"Displaying only results that are visible in the preview","docs.searchBar.popover.hasNotDownload":"Displaying results exclusively from pages shown in the preview.**Download the document to see all.**","docs.sidebar.areaLink":"Discover %1$s of %2$s %3$s","docs.suggestedArea.title":"Often downloaded together","docs.zoomLabels.zoomExpand":"Enlarge","docs.zoomLabels.zoomMaxWidth":"Maximum Width","docs.zoomLabels.zoomNormal":"Default View","docs.zoomLabels.zoomReduce":"Minimise","docs.zoomLabels.zoomThumbs":"Thumbnails","footer.country.de":"German","footer.country.en":"English","footer.country.es":"Spanish","footer.country.fr":"French","footer.country.pt":"Portuguese","footer.options.howdocsitywork":"How does Docsity work","footer.options.partners":"Partners","footer.options.sell":"Sell documents","footer.options.sellerguide":"Seller's Handbook","footer.options.support":"Contact us","footer.options.whoweare":"About us","footer.options.workwithus":"Career","footer.privacy.cookie":"Cookie Policy","footer.privacy.cookiesetup":"Cookie setup","footer.privacy.privacy":"Privacy Policy","footer.privacy.terms":"Terms of Use","footer.sitemap.biologiaechimica":"Biology and Chemistry","footer.sitemap.country":"Sitemap Languages and Countries","footer.sitemap.economia":"Economics","footer.sitemap.fisica":"Physics","footer.sitemap.giurisprudenza":"Law","footer.sitemap.ingegneria":"Engineering","footer.sitemap.lastquestions":"Latest questions","footer.sitemap.latestdoc":"Sitemap Latest Documents","footer.sitemap.lettereecomunicazione":"Literature and Communication","footer.sitemap.management":"Management","footer.sitemap.medicinaefarmacia":"Medicine and Pharmacy","footer.sitemap.psicologiaesociologia":"Psychology and Sociology","footer.sitemap.resource":"Sitemap Resources","footer.sitemap.scienzepolitiche":"Search Videos Courses and exercises carried out","footer.sitemap.storiaefilosofia":"History and Philosophy","footer.sitemap.supportopersonalizzato":"Customized support","footer.sitemap.tesinedimaturita":"High school diploma papers","footer.sitemap.topics":"Study Topics Sitemap","footer.sitemap.traccesvolteannipassati":"Proofs of previous years","footer.staticroutes.degreethesisLabel":"Thesis","footer.staticroutes.degreethesisUrl":"degree-thesis","footer.staticroutes.examquestionsLabel":"Exam","footer.staticroutes.examquestionsUrl":"exam-questions","footer.staticroutes.exceriseUrl":"exercises","footer.staticroutes.exerciseLabel":"Exercises","footer.staticroutes.notesLabel":"Lecture notes","footer.staticroutes.notesUrl":"lecture-notes","footer.staticroutes.schemesLabel":"Schemes","footer.staticroutes.schemesUrl":"schemes","footer.staticroutes.storeLabel":"Document Store","footer.staticroutes.studynotesLabel":"Study notes","footer.staticroutes.studynotesUrl":"study-notes","footer.staticroutes.summariesLabel":"Summaries","footer.staticroutes.summariesUrl":"summaries","general.docTitle":"%1$s of %2$s","generalDoc.beforeYear":"Pre %s","generalDoc.documentLabel":"Document","generalDoc.documentsLabel":"Documents","generalDoc.lessInfo":"Less info","generalDoc.moreInfo":"More info","generalDoc.orLabel":"Or","generalDoc.pointsLabel":"Points","generalDoc.shareLabel":"Share","generalDoc.titleSuffix":"%1$s of %2$s","header.common.points":"Points","header.contentArea.blogLink":"Go to the blog","header.contentArea.blogTitle":"From our blog","header.ctaUpload.sellerTooltip":"Share or sell documents","header.ctaUpload.tooltip":"Share documents","header.firstBlock.contentArea.0.heading":"Video Courses","header.firstBlock.contentArea.0.heading_pt":"Videolessons","header.firstBlock.contentArea.0.text":"Prepare yourself with lectures and tests carried out based on university programs!","header.firstBlock.contentArea.1.heading":"Find documents","header.firstBlock.contentArea.1.text":"Prepare for your exams with the study notes shared by other students like you on Docsity","header.firstBlock.contentArea.2.heading":"Search for your university","header.firstBlock.contentArea.2.text":"Find the specific documents for your university's exams","header.firstBlock.contentArea.3.heading":"Quiz","header.firstBlock.contentArea.3.text":"Respond to real exam questions and put yourself to the test","header.firstBlock.contentArea.4.link":"Search through all study resources","header.firstBlock.contentArea.6.heading":"Papers %s","header.firstBlock.contentArea.6.text":"Study with past exams, summaries and useful tips","header.firstBlock.contentArea.7.heading":"Explore questions","header.firstBlock.contentArea.7.text":"Clear up your doubts by reading the answers to questions asked by your fellow students","header.firstBlock.contentArea.8.heading":"Study topics","header.firstBlock.contentArea.8.text":"Explore the most downloaded documents for the most popular study topics","header.firstBlock.contentArea.ai":"Summarize your documents, ask them questions, convert them into quizzes and concept maps","header.firstBlock.infoArea.heading":"Prepare for your exams","header.firstBlock.infoArea.text":"Study with the several resources on Docsity","header.login":"Log in","header.menu_voices.advice":"Guidelines and tips","header.menu_voices.ai.text":"Docsity AI","header.menu_voices.ctaEvent":"Go live","header.menu_voices.earn.text":"Sell on Docsity","header.menu_voices.exam":"Prepare for your exams","header.menu_voices.points":"Get points","header.menu_voices.premium":"Premium plans","header.menu_voices.subscrive":"Subscribe","header.notifications.all_notifications":"All notifications","header.notifications.label":"Notifications","header.premium.reload":"Power top-up","header.premium.signup":"Get Premium","header.record":"Record a lesson","header.register":"Sign up","header.search.empty":"No documents found for this query","header.search.label":"Search in","header.search.placeholder":"What are you studying today?","header.search.scope.document":"Documents","header.search.scope.professor":"Professors","header.search.scope.question":"Questions","header.search.scope.quiz":"Quiz","header.search.scope.video":"Video Courses","header.secondBlock.contentArea.0.heading":"Share documents","header.secondBlock.contentArea.0.text":"For each uploaded document","header.secondBlock.contentArea.1.heading":"Answer questions","header.secondBlock.contentArea.1.text":"For each given answer (max 1 per day)","header.secondBlock.contentArea.2.link":"All the ways to get free points","header.secondBlock.contentArea.3.heading":"Get points immediately","header.secondBlock.contentArea.3.heading_premium":"Buy a Power Top-up","header.secondBlock.contentArea.3.text":"Choose a premium plan with all the points you need","header.secondBlock.contentArea.3.text_it_es":"Access all the Video Courses, get Premium Points to download the documents immediately and practice with all the Quizzes","header.secondBlock.contentArea.3.text_premium":"Get additional Premium Points that you can use until your Premium plan is active","header.secondBlock.infoArea.heading":"Earn points to download ","header.secondBlock.infoArea.text":"Earn points by helping other students or get them with a premium plan","header.thirdBlock.contentArea.0.heading":"Choose your next study program","header.thirdBlock.contentArea.0.text":"Get in touch with the best universities in the world. Search through thousands of universities and official partners","header.thirdBlock.contentArea.0.title":"Study Opportunities","header.thirdBlock.contentArea.1.title":"Community","header.thirdBlock.contentArea.2.heading":"Ask the community","header.thirdBlock.contentArea.2.text":"Ask the community for help and clear up your study doubts ","header.thirdBlock.contentArea.3.heading":"University Rankings","header.thirdBlock.contentArea.3.text":"Discover the best universities in your country according to Docsity users","header.thirdBlock.contentArea.4.heading":"Our save-the-student-ebooks!","header.thirdBlock.contentArea.4.text":"Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors","header.thirdBlock.contentArea.4.title":"Free resources","header.usermenu.ai":"Docsity AI","header.usermenu.b2b":"Study Plans","header.usermenu.chiedi_supporto":"Get support","header.usermenu.documenti":"My documents","header.usermenu.documenti_fav":"Favourites","header.usermenu.domande":"My questions","header.usermenu.downloaded":"Downloaded","header.usermenu.gestione_abbonamento":"Manage your subscription","header.usermenu.gestione_account":"Manage account","header.usermenu.homepage":"Home","header.usermenu.lezioni":"My lessons","header.usermenu.recensioni":"My reviews","header.usermenu.uploaded":"Uploaded","header.usermenu.vendite":"Sales area","header.usermenu.vendite.chip":"Balance","header.usermenu.videocorsi":"My video courses","headerLogged.searchPlaceholder":"Search for documents, lecture-notes, exam banks...","headerLogged.subscribe":"Subscribe","headerSidebar.aiStudio":"AI Studio","headerSidebar.archive":"Archive","headerSidebar.banner.description":"From audio notes, videos, links and photos to **summaries**, **maps** and **flashcards**: Docsity AI generates content that's easy for you to study.","headerSidebar.banner.title":"Try the Docsity AI app","headerSidebar.close":"Close","headerSidebar.demoTitle":"Test note","headerSidebar.downloadApp":"Download the app","headerSidebar.filters.downloaded":"Downloaded","headerSidebar.filters.emptyDownloaded":"Here, you will find all your downloaded documents","headerSidebar.filters.emptyUploaded":"Here you will find your documents uploaded from the web or the app","headerSidebar.filters.uploaded":"Uploaded","headerSidebar.itTakesNote":"AI takes notes for you","headerSidebar.landingSeller":"Sell on Docsity","headerSidebar.library":"Your library","headerSidebar.linkFavorites":"Favourites","headerSidebar.linkFavoritesEmpty":"Like a document? Click on this symbol to add it to your favourites","headerSidebar.linkHome":"Home","headerSidebar.linkLogin":"Sign up","headerSidebar.linkRecents":"Recently seen by you","headerSidebar.linkRecentsEmpty":"Here you will find your most recently viewed documents, start your search now!","headerSidebar.linkSeller":"Sales area","headerSidebar.loading":"Creation in progress...","headerSidebar.open":"Open","headerSidebar.popoverViewAll":"View all","headerSidebar.uploadCTA":"Upload","modals.modify_email_action_label":"Change email","modals.modify_email_heading":"Change you email address","modals.modify_email_placeholder":"Insert email address","modals.resend_email_action":"Request email","modals.resend_email_body":"**Before requesting the email, log in to your mailbox** to make sure it's still active","modals.resend_email_body_guest":"Before proceeding, confirm your email address. We sent you a link to confirm your email address.","modals.resend_email_heading":"Request confirmation email","modals.resend_email_invalid_confirm":"Resend confirmation email","modals.resend_email_invalid_description":"Please enter a valid email address","modals.resend_email_invalid_divider":"Or","modals.resend_email_invalid_edit":"Change email","modals.resend_email_invalid_modifica":"to keep the current one","modals.resend_email_invalid_title":"Your email address does not appear to be valid.","modals.resend_email_success_action":"Resend email","modals.resend_email_success_body":"Check if you received it at following email address: **%s**. If you don't find it in your inbox, check your spam folder.","modals.resend_email_success_title":"We sent you another email","notifications.actionButton.document.ko.review":"Read our guidelines","notifications.actionButton.points":"Find documents","notifications.actionButton.profile":"See suggestions","notifications.actionButton.review":"Leave a review","notifications.actionButton.seller":"Read the Strategies","notifications.actionButton.share":"Share","notifications.actionButtonGroup.point_blue":"Find documents","notifications.actionButtonGroup.review":"Leave a review","notifications.actionTextGroup.document_store":"Spread the word to other students like you","notifications.actionTextGroup.point_blue":"Download your favourite documents right away!","notifications.actionTextGroup.points":"Download your favourite documents right away!","notifications.content.document.change_approved":"The changes to document **%s** were approved","notifications.content.document.change_rejected":"The changes to document **%s** were not approved","notifications.content.document.ko.afterReview":"Te review of your document “%s” has not been accepted, as the document does not meet our guidelines","notifications.content.document.ko.noReview":"Your document ”%s\" wasn't published. \u003cbr/\u003e\u003cu\u003e**Read our guidelines**\u003c/u\u003e","notifications.content.document.ko.review":"Your document ”%s\" wasn't published. \u003cbr/\u003e\u003cu\u003e**Request a review**\u003c/u\u003e","notifications.content.document.store_approved":"The %s document is officially on sale at the Store","notifications.content.invoice.ready":"A new invoice is available in your personal area","notifications.content.oboarding_b2b":"We help you find **work and study opportunities that suit you:** answer a few questions and get %s Download Points","notifications.content.point_blue_answer":"You received %s Download pts for answering the question %s","notifications.content.point_blue_answer_best":"You received %s Download pts because your answer was rated as the best answer.","notifications.content.point_blue_document_download":"You received %s Download pts because %s users downloaded your document %s","notifications.content.point_blue_document_download_1":"You received %s Download pts because a user downloaded your document %s","notifications.content.point_blue_document_promoted":"You have received **%d Download points** for promoting your documents to **Top**","notifications.content.point_blue_document_request":"Some of your documents have been selected to become **Top**. Promote them and earn **%d Download points for each**","notifications.content.point_blue_premium_ai":"You have received **%d Download Points** for activating a Premium plan","notifications.content.point_blue_profile":"You received %s Download pts for completing your profile.","notifications.content.point_blue_review_document":"You received %s Download pts for leaving a review to %s","notifications.content.point_blue_review_professor":"You received %s Download points for reviewing professor %s","notifications.content.point_blue_review_university":"You received %s Download pts for leaving a review to %s","notifications.content.point_blue_upload":"You received %s Download pts for uploading the document %s","notifications.content.point_yellow.renewal":"You received %s Premium pts upon the renewal of your %s subscription.","notifications.content.profile":"Did you know that we suggest **content based on the subjects listed in your profile**? These are associated with the documents you have downloaded to date, but **you can update them whenever you want**!","notifications.content.question.answer":"%s replied to your question: %s.","notifications.content.question.answer_comment":"%s commented your answer to the question: %s.","notifications.content.question.answer_follow":"%s replied to the question you follow: %s.","notifications.content.question.comment":"%s added a comment to your question: %s.","notifications.content.review.notify_received":"You received new reviews last week!","notifications.content.review.receive_document":"%s rated your document: %s.","notifications.content.review.release_course":"What do you think of the **%s** course?","notifications.content.review.release_document":"Leave a review to %s and get %d Download points.","notifications.content.review.release_document_free":"Review the document you downloaded: %s\",2020-12-17 17:41:08","notifications.content.review.release_professor":"Do you know prof. %s? Write a review and get %d download points","notifications.content.review.release_professors":"Review your University professors and earn Download points","notifications.content.seller":"Learn now about some **Selling strategies**","notifications.content.seller.discovery":"Learn now about some **Selling strategies**","notifications.content.withdrawal.pending_fraud":"Your **withdrawals are suspended** because our payment gateway **has abnormal requests**. We are verifying that everything is ok.","notifications.contentGroup.comments":"%s and other %s users added a comment to your question: %s.","notifications.contentGroup.comments_2":"%s and another user added a comment to your question: %s.","notifications.contentGroup.docreviews_2":"You have %s pending reviews. Get %d Download points for each reviewed document","notifications.contentGroup.docreviews_free":"**%s** and other %s users replied to your question: %s.","notifications.contentGroup.documents":"%s and other %s users reviewed your document: %s.","notifications.contentGroup.documents_2":"%s and another user reviewed your document: %s.","notifications.contentGroup.follows":"%s and other %s users replied to a question you follow: %s.","notifications.contentGroup.follows_2":"%s and another user replied a question you follow: %s.","notifications.contentGroup.point_blue":"You received %s Download points since the last time.","notifications.contentGroup.point_blue_review_professor":"You received %s Download points since the last time","notifications.contentGroup.question_answers":"**%s** and other %s users replied to your question: \"%s\".","notifications.contentGroup.question_answers_2":"%s and another user replied to your question: %s.","notifications.contentGroup.questions":"%s and other %s commented your answer to the question: %s.","notifications.contentGroup.questions_2comments":"%s and another user commented your answer to the question: %s.","phoneCondition.conditionOne":"Do not use a VOIP number or a temporary number, they are not accepted","phoneCondition.conditionThree":"We will not be able to retrieve your number, so don't lose it","phoneCondition.conditionTwo":"You will need your number every time you want to withdraw","premiumAIModal.box.bottom":"Cancel anytime","premiumAIModal.box.claim":"%1$s%2$s per month","premiumAIModal.box.head.premium":"AI is already included in your subscription","premiumAIModal.close":"Close","premiumAIModal.close.download":"Continue without Premium","premiumAIModal.cta":"Subscribe now","premiumAIModal.cta.download":"Activate Premium AI","premiumAIModal.feature1":"Use AI on **private files**","premiumAIModal.feature2":"Turn **your files into summaries**","premiumAIModal.feature3":"Generate concept maps and **edit them however you like**","premiumAIModal.feature3updated":"Generate and export **concept maps**","premiumAIModal.feature4":"Generate quizzes, with **solutions included**","premiumAIModal.feature5":"Make **insights in chat**","premiumAIModal.firstPurchase":"With your first purchase, you'll receive **%s blue points** to download documents","premiumAIModal.subheading.blockCopy":"Get full access to copy, save, and use content as many times as you want.","premiumAIModal.subheading.download":"To download summaries, maps, and quizzes generated by AI, you need Premium AI.","premiumAIModal.title":"Study with AI **without limits**","premiumAIModal.title.blockCopy":"Upgrade to Premium **to copy** text","premiumAIModal.title.download":"Unlock downloads with Premium AI","promoAppModal.downloadApp":"Download the app","promoAppModal.features1":"You listen, AI writes every word for you","promoAppModal.features2":"Scan your notes and books","promoAppModal.features3":"Create summaries, maps and quizzes with AI","promoAppModal.title":"Download the app and transcribe lessons with AI","shareModal.copyAction":"Copy","shareModal.linkCopied":"Link copied!","shareModal.title":"Share \"%s\"","uploadModal.alertCapReached":"You've reached the daily limit of %1$d document uploads for using the AI. Come back tomorrow to upload more documents.","uploadModal.alertDocumentSize":"The file is too large: how about compressing and re-uploading it? ( %d MB is the maximum size)","uploadModal.alertLoadFailed":"Oops! Something went wrong","uploadModal.alertPageCount":"Currently we cannot handle files that have less than %1$s pages or more than %2$s pages.","uploadModal.community.buttonTitle":"Upload file","uploadModal.community.capReached.features1":"AI Studio not available","uploadModal.community.capReached.features2":"Earn points by helping others","uploadModal.community.features1":"Earn points by helping others","uploadModal.community.features2":"Use AI Studio Basic for free","uploadModal.community.features3":"Earn points by helping others","uploadModal.community.featuresRemaining":"Use AI Studio for free (%1$s %2$s)","uploadModal.community.points":"points","uploadModal.community.premium.features1":"Process your content with AI","uploadModal.community.title":"Upload to the community","uploadModal.community.usePlural":"remaining uses","uploadModal.community.useSingular":"remaining use","uploadModal.dropArea":"Drag and drop files","uploadModal.maxSizeTip":"PDFs of max %1$s MB that have between %2$s and %3$s pages","uploadModal.private.features1":"Upload documents that are only visible to you","uploadModal.private.features2":"Use AI Studio without limits","uploadModal.private.features3":"Included in your subscription","uploadModal.private.title":"Upload privately","uploadModal.selectButton":"Choose file","uploadModal.sellerBox.becomeSeller":"\u003cu\u003e**Become a seller!**\u003c/u\u003e","uploadModal.sellerBox.description":"Do you want to make money with your notes?","uploadModal.sellerBox.isSeller":"\u003cu\u003e**Sell Documents**\u003c/u\u003e","uploadModal.title":"We're uploading the document","uploadModal.uploadArea":"Upload your file","uploadModal.uploadTypeLimitations":"Photos and scans are not compatible","uploadModal.wrongFile":"Wrong type of file","word.common.documenti":"Documents","word.common.domande":"questions","word.common.maturita":"maturity","word.common.quiz":"Quiz","word.common.test":"test","word.common.universita":"University","word.common.veditutte":"View all","word.common.veditutti":"View all","word.common.video":"Video Courses","default":{"alerts.action_spacer":"or","alerts.back_lang.action":"Back to %s language","alerts.complete_profile.link":"Complete profile","alerts.complete_profile.text":"Help us recommend the best content for you and receive instantly **%s download points**","alerts.confirm_email.action_modifica":"Change email address","alerts.confirm_email.action_reinvia":"Resend email.","alerts.confirm_email.text":"Confirm your email address by clicking on the link we sent you at %s.","alerts.diff_lang.action":"Click here","alerts.diff_lang.text":"You are browsing in a language other than your own","alerts.invalid_email.action_modifica":"Change email","alerts.invalid_email.action_resend":"Resend email","alerts.invalid_email.text":"We are unable to send the confirmation email to your address: **%s**","alerts.low_points.action":"Get points immediately","alerts.low_points.main":"You are running out of Download Points.","common.date.formatAcademicYear":"Pre %s","completeProfile.form_label":"Birth year","completeProfile.form_label_cta":"Confirm data","completeProfile.label_cta":"Start now","completeProfile.text_default_1":"You'll earn % points","completeProfile.text_default_2":"to download documents and gain access to free Video Courses and Quizzes","completeProfile.text_default_3":"to download documents","completeProfile.text_guest":"To proceed, enter the missing data","completeProfile.title_default":"Update your profile to continue","completeProfile.title_guest":"Oops, looks like you missed something!","docProfessorModal.disclaimer":"The page is generated automatically and is not managed by a person (the Professor or anyone else). The documents listed are not official or representative of the teaching methods and contents of the Professor in question. They are automatically aggregated according to criteria entered by users. The personal data relating to the Professor contained in this page have been collected by the Data Controller (Ladybird s.r.l.) from open sources accessible to the public and / or have been generated directly by the users. The goal is to enrich the contents of the platform for the legitimate interest of Ladybird s.r.l. to improve its usability for the benefit of its users. They will be stored until any opposition to the processing by the interested party, who is entitled to oppose the processing for reasons related to their particular situation, except for the provisions of art. 21 § 1, Paragraph 2, GDPR. For more information on the categories of recipients of the personal data and the additional rights of the data subject, please refer to sections B and H respectively of the Docsity privacy policy available at this [link](%s)","docs.abstractArea.documents":"Documents","docs.abstractArea.seoString":"Download %1$s and more %2$s %3$s in PDF only on Docsity!","docs.abstractArea.thisSubject":"This subject","docs.abstractArea.title":"Partial preview of the text","docs.actionSection.disabledByLanguageUser":"Only %s Docsity users can download this document","docs.actionSection.disabledByUserNotSeller":"We have received reports about this document. The download is temporarily disabled.","docs.actions.addFavourite":"Add to favourites","docs.actions.approveDocument":"Approve Document","docs.actions.approveDocument.force":"Force document approval","docs.actions.cannotEditPrime":"This document can no longer be modified because it has been promoted as a Top Document","docs.actions.deleteDocument":"Delete","docs.actions.download":"Download","docs.actions.downloadDocuemnt":"Download the document","docs.actions.edit":"Edit","docs.actions.follow":"Follow","docs.actions.reconvertDocument.errorMessage":"Error during the request","docs.actions.reconvertDocument.label":"Convert the document","docs.actions.reconvertDocument.successMessage":"Conversion successfully completed","docs.actions.remove":"Remove","docs.actions.removeFavourite":"Remove from favourites","docs.actions.reportDocument":"Report document","docs.actions.requestError":"An error has occurred","docs.actions.requestSuccess":"Request submitted, wait a few seconds and reload the page","docs.actions.review":"Leave a review","docs.actions.save":"Save","docs.actions.shareDocTooltip":"Share document","docs.actions.shareLabel":"Share","docs.actions.unfollow":"Unfollow","docs.actions.useAI":"Try Docsity AI","docs.actions.watchReview":"Your review","docs.aiOffCanvas.aiAlertFreemium":"**Free trial on %s documents or use without limits with Premium**","docs.aiOffCanvas.aiCtaDownload":"Download document","docs.aiOffCanvas.aiDownload":"You can use AI after downloading the document","docs.aiOffCanvas.aiDownloadDocument":"**AI is available with a Premium plan.** You can use AI after downloading the document, no additional cost.","docs.aiOffCanvas.aiFeaturePoints":"%s points","docs.aiOffCanvas.chat.infoPoint1":"Explore the content of a document with chat responses","docs.aiOffCanvas.chat.infoPoint2":"Clear any doubts to study in less time","docs.aiOffCanvas.chat.title":"Ask the document","docs.aiOffCanvas.map.infoPoint1":"Get the conceptual map of a document, generated by the AI","docs.aiOffCanvas.map.infoPoint2":"Edit the map","docs.aiOffCanvas.map.infoPoint3":"Download a PNG","docs.aiOffCanvas.map.title":"Concept map","docs.aiOffCanvas.points":"points","docs.aiOffCanvas.quiz.infoPoint1":"Generate a number of questions on the topics covered by the document","docs.aiOffCanvas.quiz.infoPoint2":"Download the PDF quiz so you can study whenever you want","docs.aiOffCanvas.quiz.infoPoint3":"Get the correct answer for each question","docs.aiOffCanvas.quiz.title":"Quiz","docs.aiOffCanvas.subtitle":"What you can get","docs.aiOffCanvas.summary.infoPoint1":"Summary of approximately 70%, in just a few minutes and in PDF format","docs.aiOffCanvas.summary.infoPoint1updated":"Summarise the document in a few minutes","docs.aiOffCanvas.summary.infoPoint2":"Download the summary document in PDF format","docs.aiOffCanvas.summary.infoPoint2updated":"Download the summary in PDF format","docs.aiOffCanvas.summary.title":"Summary","docs.aiOffCanvas.title":"Use Docsity AI on the document","docs.aiOffCanvas.titleUpdated":"Download the document to use AI","docs.breadcrumb.free":"Documents","docs.breadcrumb.store":"Store","docs.ctaArea.toggleSidebarTooltip":"Find here **related documents** and other useful resources","docs.ctaMessages.AI":"AI","docs.ctaMessages.alreadyDownloadedTitle":"You already downloaded this document","docs.ctaMessages.alreadyReviewedTitle":"You've already downloaded and reviewed this document","docs.ctaMessages.descriptionAI":"Make a summary, create a concept map or a quiz from this document","docs.ctaMessages.disabledByLanguageUser":"Only %s Docsity users can download this document","docs.ctaMessages.disabledByUserNotSeller":"This document is temporarily unavailable for download","docs.ctaMessages.discountBadge":"On special offer","docs.ctaMessages.myDocumentOffer":"Your document is discounted temporarily","docs.ctaMessages.uploadedTtile":"You uploaded this document on %s","docs.docDeletionModal.empty":"The field is mandatory","docs.docDeletionModal.notice":"Document_notice","docs.docDeletionModal.report":"Reports","docs.docDeletionModal.submit":"Delete Document","docs.docDeletionModal.textArea.label":"Notes","docs.docDeletionModal.textArea.optional":"(optional)","docs.docDeletionModal.textArea.placeholder":"Write here...","docs.docDeletionModal.title":"Select the reason for deletion","docs.docMyDiscountModal.close":"OK, close","docs.docMyDiscountModal.content":"Since this document was not receiving downloads, we decided to discount it to. When the number of downloads goes back up, we will increase the price again.","docs.docMyDiscountModal.title":"This document has been discounted","docs.docProfessorModal.disclaimer":"The page is generated automatically and is not managed by a person (the Professor or anyone else). The documents listed are not official or representative of the teaching methods and contents of the Professor in question. They are automatically aggregated according to criteria entered by users. The personal data relating to the Professor contained in this page have been collected by the Data Controller (Ladybird s.r.l.) from open sources accessible to the public and / or have been generated directly by the users. The goal is to enrich the contents of the platform for the legitimate interest of Ladybird s.r.l. to improve its usability for the benefit of its users. They will be stored until any opposition to the processing by the interested party, who is entitled to oppose the processing for reasons related to their particular situation, except for the provisions of art. 21 § 1, Paragraph 2, GDPR. For more information on the categories of recipients of the personal data and the additional rights of the data subject, please refer to sections B and H respectively of the Docsity privacy policy available at this [link](%s)","docs.docReportingModal.email":"Email","docs.docReportingModal.errorMessage":"We received a report from you regarding this document on %1$s with reason *“%2$s“*","docs.docReportingModal.errorTitle":"You have reported this document before","docs.docReportingModal.genericMessage":"We will try to solve this as quickly as possible.","docs.docReportingModal.guestName":"Name and surname","docs.docReportingModal.leaveAmessage":"Leave a message","docs.docReportingModal.optional":"Optional","docs.docReportingModal.reasons.copyrightViolation":"This document contains copyright infringement","docs.docReportingModal.reasons.duplicatedDocument":"This document has been duplicated","docs.docReportingModal.reasons.inconsitentContent":"The content is not consistent with the description","docs.docReportingModal.reasons.uploadedMyDocument":"User has uploaded a document that belongs to me","docs.docReportingModal.required":"Field required","docs.docReportingModal.sendReport":"Send report","docs.docReportingModal.successMessage":"Error processing your request","docs.docReportingModal.successTitle":"Thanks for reporting","docs.docReportingModal.title":"Why do you want to report this document?","docs.docReportingModal.validationError.fieldTooLong":"The max. number of characters is %d","docs.download.idle.buttonLabel":"Download now","docs.download.idle.label":"Download using","docs.download.idle.label2":"It will be available on all devices","docs.download.idle.selectDownload":"%s download Points","docs.download.idle.selectPremium":"%s premium Points","docs.download.idle.title":"You're downloading...","docs.download.noPoints.ctaTitle":"Download now","docs.download.noPoints.powerCtaSubtitle":"by purchasing a Power Recharge","docs.download.noPoints.premiumCtaSubtitle":"by purchasing a Premium plan","docs.download.noPoints.premiumUploadAction":"[Upload documents](%s) and get download points within a **few minutes**","docs.download.noPoints.rewardLabel":"and get the points you are missing","docs.download.noPoints.rewardTimeLabel":"*%s* Points are awarded within a **few minutes**","docs.download.noPoints.title":"Your points:","docs.download.noPoints.uploadAction":"Share documents","docs.download.redirecting.close":"Close","docs.download.redirecting.title":"There was a problem, [try again](%s)","docs.download.resend_email_action":"Resend email","docs.download.resend_email_body":"Before proceeding confirm your email. Click the link that we just sent you at **%s**.","docs.download.resend_email_heading":"Oops! Something is missing to download the document","docs.download.resend_email_invalid_description":"We can't deliver messages to **%s**. Before proceeding, double check whether your email address is valid or you need to change it.","docs.download.resend_email_invalid_title":"Oops! Something is missing to download the document","docs.flatSection.academicYear":"Academic Year","docs.flatSection.description":"Description","docs.flatSection.download":"Download","docs.flatSection.favourites":"Favourites","docs.flatSection.follow":"Follow","docs.flatSection.hide":"Hide","docs.flatSection.multipleDocLabel":"Documents","docs.flatSection.multipleReviewsLabel":"Reviews","docs.flatSection.page":"Page","docs.flatSection.pageNumber":"Number of pages","docs.flatSection.pages":"Pages","docs.flatSection.professorPrefix":"Prof. %s","docs.flatSection.sellDateLabel":"Available from","docs.flatSection.showMore":"Show more","docs.flatSection.singleDocLabel":"Document","docs.flatSection.singleReviewsLabel":"Review","docs.flatSection.unfollow":"Unfollow","docs.flatSection.uploadDateLabel":"Uploaded on","docs.header.alreadyDownloadedTitle":"You already downloaded this document","docs.header.alreadyReviewedTitle":"You've already downloaded and reviewed this document","docs.header.review":"%d Review","docs.header.reviews":"%d Reviews","docs.header.uploadedTtile":"You uploaded this document on %s","docs.headingArea.notPublished":"Document not published","docs.headingArea.topTooltip":"One of the most popular and successful documents in our community","docs.hero.multipleDocLabel":"Documents","docs.hero.notPublished":"Document not published","docs.hero.professorPrefix":"Prof. %s","docs.hero.questionsLabel":"What you will learn","docs.hero.review":"%d Review","docs.hero.reviews":"%d Reviews","docs.hero.sellDateLabel":"Available from","docs.hero.singleDocLabel":"Document","docs.hero.typology":"Typology: %s","docs.hero.unknownUser":"unknown user","docs.hero.uploadDateLabel":"Uploaded on","docs.infoArea.alreadyReviewedTitle":"You already downloaded this document","docs.infoArea.uploadedTtile":"You uploaded this document on %s","docs.intentClose.action":"Show others","docs.intentClose.heading":"Abracadabra 🔮 More documents for you! There’s really no magic, only our massive library!","docs.player.controllerLabel":"Page %1$d / %2$d","docs.player.crossPaywall.buttonPremiumLabel":"Download now","docs.player.crossPaywall.buttonShareLabel":"Share documents","docs.player.crossPaywall.textButtonPremium":"by purchasing a Premium plan","docs.player.crossPaywall.textButtonShare":"and get the points you are missing in **%s hours**","docs.player.downloadBlock.always.buttonLabel":"Download for free","docs.player.downloadBlock.always.copy":"This page cannot be seen from the preview","docs.player.downloadBlock.always.copyLastPage":"You can download it any time from any device","docs.player.downloadBlock.always.title":"You already downloaded this document","docs.player.downloadBlock.always.titleLastPage":"You already downloaded this document","docs.player.downloadBlock.default.buttonLabel":"Download","docs.player.downloadBlock.default.copy":"This page cannot be seen from the preview","docs.player.downloadBlock.default.copyLastPage":"You can download it any time from any device","docs.player.downloadBlock.default.title":"Don't miss anything!","docs.player.downloadBlock.default.titleLastPage":"Download the full document","docs.player.paywallBlock.unlock.buttonLabel":"Read the document","docs.player.paywallBlock.unlock.title":"Register for free to read the full document","docs.player.previewLabel":"Document preview","docs.playerToolbar.pageNumber":"Number of pages","docs.playerToolbar.scrollTop":"Go back to top","docs.priceModal.pointsImgAlt":"Get points","docs.priceModal.pointsLinkLabel":"Other ways to get download points for free","docs.priceModal.pointsSubtitle":"[Upload](%1$s) your documents or [answer](%2$s) questions and get download points","docs.priceModal.pointsTitle":"Get points to download the document","docs.priceModal.premiumImgAlt":"Go Premium","docs.priceModal.premiumLinkLabel":"See our Premium plans","docs.priceModal.premiumSubtitle":"Choose one of our Premium plans and use the points to download your documents right away","docs.priceModal.premiumTitle":"Don't want to wait?","docs.priceSection.discountBadge":"On special offer","docs.priceSection.discountLimitedLabel":"Limited-time offer","docs.priceSection.points":"Points","docs.recentArea.emptyPlaceholder":"Here you'll find the latest visited documents","docs.recentArea.title":"Recently viewed documents","docs.relatedArea.emptyPlaceholder":"There are no similar documents","docs.relatedArea.title":"Related documents","docs.relatedArea.viewOthers":"Show others","docs.reviewsArea.noReviewTitle":"No reviews yet","docs.reviewsArea.title":"Reviews","docs.reviewsArea.viewAll":"View all","docs.reviewsModal.awaitingModeration":"Under moderation","docs.reviewsModal.errorCase":"Something went wrong","docs.reviewsModal.getPoints":"Get %d download points","docs.reviewsModal.leaveReview":"Leave a review","docs.reviewsModal.orderHighest":"Highest rate","docs.reviewsModal.orderLowest":"Lowest rate","docs.reviewsModal.orderRecent":"Most recent","docs.reviewsModal.rating1":"Poor","docs.reviewsModal.rating2":"Insufficient","docs.reviewsModal.rating3":"Sufficient","docs.reviewsModal.rating4":"Good","docs.reviewsModal.rating5":"Excellent","docs.reviewsModal.reload":"Top-up","docs.reviewsModal.showMore":"Show others","docs.reviewsModal.showRatingDetails":"Show more","docs.reviewsModal.sortBy":"Sort by","docs.reviewsModal.title":"Reviews","docs.reviewsModal.whoCanReview":"Only users who downloaded the document can leave a review","docs.reviewsModal.yourReview":"Your review","docs.searchBar.counter":"%(index)s of %(total)s **(%(matches)s visible)**","docs.searchBar.error.button":"Reload","docs.searchBar.error.content":"Try searching again","docs.searchBar.error.title":"Loading error","docs.searchBar.placeholder":"Search in the preview","docs.searchBar.popover.close":"Close","docs.searchBar.popover.download":"Download","docs.searchBar.popover.hasDownload":"Displaying only results that are visible in the preview","docs.searchBar.popover.hasNotDownload":"Displaying results exclusively from pages shown in the preview.**Download the document to see all.**","docs.sidebar.areaLink":"Discover %1$s of %2$s %3$s","docs.suggestedArea.title":"Often downloaded together","docs.zoomLabels.zoomExpand":"Enlarge","docs.zoomLabels.zoomMaxWidth":"Maximum Width","docs.zoomLabels.zoomNormal":"Default View","docs.zoomLabels.zoomReduce":"Minimise","docs.zoomLabels.zoomThumbs":"Thumbnails","footer.country.de":"German","footer.country.en":"English","footer.country.es":"Spanish","footer.country.fr":"French","footer.country.pt":"Portuguese","footer.options.howdocsitywork":"How does Docsity work","footer.options.partners":"Partners","footer.options.sell":"Sell documents","footer.options.sellerguide":"Seller's Handbook","footer.options.support":"Contact us","footer.options.whoweare":"About us","footer.options.workwithus":"Career","footer.privacy.cookie":"Cookie Policy","footer.privacy.cookiesetup":"Cookie setup","footer.privacy.privacy":"Privacy Policy","footer.privacy.terms":"Terms of Use","footer.sitemap.biologiaechimica":"Biology and Chemistry","footer.sitemap.country":"Sitemap Languages and Countries","footer.sitemap.economia":"Economics","footer.sitemap.fisica":"Physics","footer.sitemap.giurisprudenza":"Law","footer.sitemap.ingegneria":"Engineering","footer.sitemap.lastquestions":"Latest questions","footer.sitemap.latestdoc":"Sitemap Latest Documents","footer.sitemap.lettereecomunicazione":"Literature and Communication","footer.sitemap.management":"Management","footer.sitemap.medicinaefarmacia":"Medicine and Pharmacy","footer.sitemap.psicologiaesociologia":"Psychology and Sociology","footer.sitemap.resource":"Sitemap Resources","footer.sitemap.scienzepolitiche":"Search Videos Courses and exercises carried out","footer.sitemap.storiaefilosofia":"History and Philosophy","footer.sitemap.supportopersonalizzato":"Customized support","footer.sitemap.tesinedimaturita":"High school diploma papers","footer.sitemap.topics":"Study Topics Sitemap","footer.sitemap.traccesvolteannipassati":"Proofs of previous years","footer.staticroutes.degreethesisLabel":"Thesis","footer.staticroutes.degreethesisUrl":"degree-thesis","footer.staticroutes.examquestionsLabel":"Exam","footer.staticroutes.examquestionsUrl":"exam-questions","footer.staticroutes.exceriseUrl":"exercises","footer.staticroutes.exerciseLabel":"Exercises","footer.staticroutes.notesLabel":"Lecture notes","footer.staticroutes.notesUrl":"lecture-notes","footer.staticroutes.schemesLabel":"Schemes","footer.staticroutes.schemesUrl":"schemes","footer.staticroutes.storeLabel":"Document Store","footer.staticroutes.studynotesLabel":"Study notes","footer.staticroutes.studynotesUrl":"study-notes","footer.staticroutes.summariesLabel":"Summaries","footer.staticroutes.summariesUrl":"summaries","general.docTitle":"%1$s of %2$s","generalDoc.beforeYear":"Pre %s","generalDoc.documentLabel":"Document","generalDoc.documentsLabel":"Documents","generalDoc.lessInfo":"Less info","generalDoc.moreInfo":"More info","generalDoc.orLabel":"Or","generalDoc.pointsLabel":"Points","generalDoc.shareLabel":"Share","generalDoc.titleSuffix":"%1$s of %2$s","header.common.points":"Points","header.contentArea.blogLink":"Go to the blog","header.contentArea.blogTitle":"From our blog","header.ctaUpload.sellerTooltip":"Share or sell documents","header.ctaUpload.tooltip":"Share documents","header.firstBlock.contentArea.0.heading":"Video Courses","header.firstBlock.contentArea.0.heading_pt":"Videolessons","header.firstBlock.contentArea.0.text":"Prepare yourself with lectures and tests carried out based on university programs!","header.firstBlock.contentArea.1.heading":"Find documents","header.firstBlock.contentArea.1.text":"Prepare for your exams with the study notes shared by other students like you on Docsity","header.firstBlock.contentArea.2.heading":"Search for your university","header.firstBlock.contentArea.2.text":"Find the specific documents for your university's exams","header.firstBlock.contentArea.3.heading":"Quiz","header.firstBlock.contentArea.3.text":"Respond to real exam questions and put yourself to the test","header.firstBlock.contentArea.4.link":"Search through all study resources","header.firstBlock.contentArea.6.heading":"Papers %s","header.firstBlock.contentArea.6.text":"Study with past exams, summaries and useful tips","header.firstBlock.contentArea.7.heading":"Explore questions","header.firstBlock.contentArea.7.text":"Clear up your doubts by reading the answers to questions asked by your fellow students","header.firstBlock.contentArea.8.heading":"Study topics","header.firstBlock.contentArea.8.text":"Explore the most downloaded documents for the most popular study topics","header.firstBlock.contentArea.ai":"Summarize your documents, ask them questions, convert them into quizzes and concept maps","header.firstBlock.infoArea.heading":"Prepare for your exams","header.firstBlock.infoArea.text":"Study with the several resources on Docsity","header.login":"Log in","header.menu_voices.advice":"Guidelines and tips","header.menu_voices.ai.text":"Docsity AI","header.menu_voices.ctaEvent":"Go live","header.menu_voices.earn.text":"Sell on Docsity","header.menu_voices.exam":"Prepare for your exams","header.menu_voices.points":"Get points","header.menu_voices.premium":"Premium plans","header.menu_voices.subscrive":"Subscribe","header.notifications.all_notifications":"All notifications","header.notifications.label":"Notifications","header.premium.reload":"Power top-up","header.premium.signup":"Get Premium","header.record":"Record a lesson","header.register":"Sign up","header.search.empty":"No documents found for this query","header.search.label":"Search in","header.search.placeholder":"What are you studying today?","header.search.scope.document":"Documents","header.search.scope.professor":"Professors","header.search.scope.question":"Questions","header.search.scope.quiz":"Quiz","header.search.scope.video":"Video Courses","header.secondBlock.contentArea.0.heading":"Share documents","header.secondBlock.contentArea.0.text":"For each uploaded document","header.secondBlock.contentArea.1.heading":"Answer questions","header.secondBlock.contentArea.1.text":"For each given answer (max 1 per day)","header.secondBlock.contentArea.2.link":"All the ways to get free points","header.secondBlock.contentArea.3.heading":"Get points immediately","header.secondBlock.contentArea.3.heading_premium":"Buy a Power Top-up","header.secondBlock.contentArea.3.text":"Choose a premium plan with all the points you need","header.secondBlock.contentArea.3.text_it_es":"Access all the Video Courses, get Premium Points to download the documents immediately and practice with all the Quizzes","header.secondBlock.contentArea.3.text_premium":"Get additional Premium Points that you can use until your Premium plan is active","header.secondBlock.infoArea.heading":"Earn points to download ","header.secondBlock.infoArea.text":"Earn points by helping other students or get them with a premium plan","header.thirdBlock.contentArea.0.heading":"Choose your next study program","header.thirdBlock.contentArea.0.text":"Get in touch with the best universities in the world. Search through thousands of universities and official partners","header.thirdBlock.contentArea.0.title":"Study Opportunities","header.thirdBlock.contentArea.1.title":"Community","header.thirdBlock.contentArea.2.heading":"Ask the community","header.thirdBlock.contentArea.2.text":"Ask the community for help and clear up your study doubts ","header.thirdBlock.contentArea.3.heading":"University Rankings","header.thirdBlock.contentArea.3.text":"Discover the best universities in your country according to Docsity users","header.thirdBlock.contentArea.4.heading":"Our save-the-student-ebooks!","header.thirdBlock.contentArea.4.text":"Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors","header.thirdBlock.contentArea.4.title":"Free resources","header.usermenu.ai":"Docsity AI","header.usermenu.b2b":"Study Plans","header.usermenu.chiedi_supporto":"Get support","header.usermenu.documenti":"My documents","header.usermenu.documenti_fav":"Favourites","header.usermenu.domande":"My questions","header.usermenu.downloaded":"Downloaded","header.usermenu.gestione_abbonamento":"Manage your subscription","header.usermenu.gestione_account":"Manage account","header.usermenu.homepage":"Home","header.usermenu.lezioni":"My lessons","header.usermenu.recensioni":"My reviews","header.usermenu.uploaded":"Uploaded","header.usermenu.vendite":"Sales area","header.usermenu.vendite.chip":"Balance","header.usermenu.videocorsi":"My video courses","headerLogged.searchPlaceholder":"Search for documents, lecture-notes, exam banks...","headerLogged.subscribe":"Subscribe","headerSidebar.aiStudio":"AI Studio","headerSidebar.archive":"Archive","headerSidebar.banner.description":"From audio notes, videos, links and photos to **summaries**, **maps** and **flashcards**: Docsity AI generates content that's easy for you to study.","headerSidebar.banner.title":"Try the Docsity AI app","headerSidebar.close":"Close","headerSidebar.demoTitle":"Test note","headerSidebar.downloadApp":"Download the app","headerSidebar.filters.downloaded":"Downloaded","headerSidebar.filters.emptyDownloaded":"Here, you will find all your downloaded documents","headerSidebar.filters.emptyUploaded":"Here you will find your documents uploaded from the web or the app","headerSidebar.filters.uploaded":"Uploaded","headerSidebar.itTakesNote":"AI takes notes for you","headerSidebar.landingSeller":"Sell on Docsity","headerSidebar.library":"Your library","headerSidebar.linkFavorites":"Favourites","headerSidebar.linkFavoritesEmpty":"Like a document? Click on this symbol to add it to your favourites","headerSidebar.linkHome":"Home","headerSidebar.linkLogin":"Sign up","headerSidebar.linkRecents":"Recently seen by you","headerSidebar.linkRecentsEmpty":"Here you will find your most recently viewed documents, start your search now!","headerSidebar.linkSeller":"Sales area","headerSidebar.loading":"Creation in progress...","headerSidebar.open":"Open","headerSidebar.popoverViewAll":"View all","headerSidebar.uploadCTA":"Upload","modals.modify_email_action_label":"Change email","modals.modify_email_heading":"Change you email address","modals.modify_email_placeholder":"Insert email address","modals.resend_email_action":"Request email","modals.resend_email_body":"**Before requesting the email, log in to your mailbox** to make sure it's still active","modals.resend_email_body_guest":"Before proceeding, confirm your email address. We sent you a link to confirm your email address.","modals.resend_email_heading":"Request confirmation email","modals.resend_email_invalid_confirm":"Resend confirmation email","modals.resend_email_invalid_description":"Please enter a valid email address","modals.resend_email_invalid_divider":"Or","modals.resend_email_invalid_edit":"Change email","modals.resend_email_invalid_modifica":"to keep the current one","modals.resend_email_invalid_title":"Your email address does not appear to be valid.","modals.resend_email_success_action":"Resend email","modals.resend_email_success_body":"Check if you received it at following email address: **%s**. If you don't find it in your inbox, check your spam folder.","modals.resend_email_success_title":"We sent you another email","notifications.actionButton.document.ko.review":"Read our guidelines","notifications.actionButton.points":"Find documents","notifications.actionButton.profile":"See suggestions","notifications.actionButton.review":"Leave a review","notifications.actionButton.seller":"Read the Strategies","notifications.actionButton.share":"Share","notifications.actionButtonGroup.point_blue":"Find documents","notifications.actionButtonGroup.review":"Leave a review","notifications.actionTextGroup.document_store":"Spread the word to other students like you","notifications.actionTextGroup.point_blue":"Download your favourite documents right away!","notifications.actionTextGroup.points":"Download your favourite documents right away!","notifications.content.document.change_approved":"The changes to document **%s** were approved","notifications.content.document.change_rejected":"The changes to document **%s** were not approved","notifications.content.document.ko.afterReview":"Te review of your document “%s” has not been accepted, as the document does not meet our guidelines","notifications.content.document.ko.noReview":"Your document ”%s\" wasn't published. \u003cbr/\u003e\u003cu\u003e**Read our guidelines**\u003c/u\u003e","notifications.content.document.ko.review":"Your document ”%s\" wasn't published. \u003cbr/\u003e\u003cu\u003e**Request a review**\u003c/u\u003e","notifications.content.document.store_approved":"The %s document is officially on sale at the Store","notifications.content.invoice.ready":"A new invoice is available in your personal area","notifications.content.oboarding_b2b":"We help you find **work and study opportunities that suit you:** answer a few questions and get %s Download Points","notifications.content.point_blue_answer":"You received %s Download pts for answering the question %s","notifications.content.point_blue_answer_best":"You received %s Download pts because your answer was rated as the best answer.","notifications.content.point_blue_document_download":"You received %s Download pts because %s users downloaded your document %s","notifications.content.point_blue_document_download_1":"You received %s Download pts because a user downloaded your document %s","notifications.content.point_blue_document_promoted":"You have received **%d Download points** for promoting your documents to **Top**","notifications.content.point_blue_document_request":"Some of your documents have been selected to become **Top**. Promote them and earn **%d Download points for each**","notifications.content.point_blue_premium_ai":"You have received **%d Download Points** for activating a Premium plan","notifications.content.point_blue_profile":"You received %s Download pts for completing your profile.","notifications.content.point_blue_review_document":"You received %s Download pts for leaving a review to %s","notifications.content.point_blue_review_professor":"You received %s Download points for reviewing professor %s","notifications.content.point_blue_review_university":"You received %s Download pts for leaving a review to %s","notifications.content.point_blue_upload":"You received %s Download pts for uploading the document %s","notifications.content.point_yellow.renewal":"You received %s Premium pts upon the renewal of your %s subscription.","notifications.content.profile":"Did you know that we suggest **content based on the subjects listed in your profile**? These are associated with the documents you have downloaded to date, but **you can update them whenever you want**!","notifications.content.question.answer":"%s replied to your question: %s.","notifications.content.question.answer_comment":"%s commented your answer to the question: %s.","notifications.content.question.answer_follow":"%s replied to the question you follow: %s.","notifications.content.question.comment":"%s added a comment to your question: %s.","notifications.content.review.notify_received":"You received new reviews last week!","notifications.content.review.receive_document":"%s rated your document: %s.","notifications.content.review.release_course":"What do you think of the **%s** course?","notifications.content.review.release_document":"Leave a review to %s and get %d Download points.","notifications.content.review.release_document_free":"Review the document you downloaded: %s\",2020-12-17 17:41:08","notifications.content.review.release_professor":"Do you know prof. %s? Write a review and get %d download points","notifications.content.review.release_professors":"Review your University professors and earn Download points","notifications.content.seller":"Learn now about some **Selling strategies**","notifications.content.seller.discovery":"Learn now about some **Selling strategies**","notifications.content.withdrawal.pending_fraud":"Your **withdrawals are suspended** because our payment gateway **has abnormal requests**. We are verifying that everything is ok.","notifications.contentGroup.comments":"%s and other %s users added a comment to your question: %s.","notifications.contentGroup.comments_2":"%s and another user added a comment to your question: %s.","notifications.contentGroup.docreviews_2":"You have %s pending reviews. Get %d Download points for each reviewed document","notifications.contentGroup.docreviews_free":"**%s** and other %s users replied to your question: %s.","notifications.contentGroup.documents":"%s and other %s users reviewed your document: %s.","notifications.contentGroup.documents_2":"%s and another user reviewed your document: %s.","notifications.contentGroup.follows":"%s and other %s users replied to a question you follow: %s.","notifications.contentGroup.follows_2":"%s and another user replied a question you follow: %s.","notifications.contentGroup.point_blue":"You received %s Download points since the last time.","notifications.contentGroup.point_blue_review_professor":"You received %s Download points since the last time","notifications.contentGroup.question_answers":"**%s** and other %s users replied to your question: \"%s\".","notifications.contentGroup.question_answers_2":"%s and another user replied to your question: %s.","notifications.contentGroup.questions":"%s and other %s commented your answer to the question: %s.","notifications.contentGroup.questions_2comments":"%s and another user commented your answer to the question: %s.","phoneCondition.conditionOne":"Do not use a VOIP number or a temporary number, they are not accepted","phoneCondition.conditionThree":"We will not be able to retrieve your number, so don't lose it","phoneCondition.conditionTwo":"You will need your number every time you want to withdraw","premiumAIModal.box.bottom":"Cancel anytime","premiumAIModal.box.claim":"%1$s%2$s per month","premiumAIModal.box.head.premium":"AI is already included in your subscription","premiumAIModal.close":"Close","premiumAIModal.close.download":"Continue without Premium","premiumAIModal.cta":"Subscribe now","premiumAIModal.cta.download":"Activate Premium AI","premiumAIModal.feature1":"Use AI on **private files**","premiumAIModal.feature2":"Turn **your files into summaries**","premiumAIModal.feature3":"Generate concept maps and **edit them however you like**","premiumAIModal.feature3updated":"Generate and export **concept maps**","premiumAIModal.feature4":"Generate quizzes, with **solutions included**","premiumAIModal.feature5":"Make **insights in chat**","premiumAIModal.firstPurchase":"With your first purchase, you'll receive **%s blue points** to download documents","premiumAIModal.subheading.blockCopy":"Get full access to copy, save, and use content as many times as you want.","premiumAIModal.subheading.download":"To download summaries, maps, and quizzes generated by AI, you need Premium AI.","premiumAIModal.title":"Study with AI **without limits**","premiumAIModal.title.blockCopy":"Upgrade to Premium **to copy** text","premiumAIModal.title.download":"Unlock downloads with Premium AI","promoAppModal.downloadApp":"Download the app","promoAppModal.features1":"You listen, AI writes every word for you","promoAppModal.features2":"Scan your notes and books","promoAppModal.features3":"Create summaries, maps and quizzes with AI","promoAppModal.title":"Download the app and transcribe lessons with AI","shareModal.copyAction":"Copy","shareModal.linkCopied":"Link copied!","shareModal.title":"Share \"%s\"","uploadModal.alertCapReached":"You've reached the daily limit of %1$d document uploads for using the AI. Come back tomorrow to upload more documents.","uploadModal.alertDocumentSize":"The file is too large: how about compressing and re-uploading it? ( %d MB is the maximum size)","uploadModal.alertLoadFailed":"Oops! Something went wrong","uploadModal.alertPageCount":"Currently we cannot handle files that have less than %1$s pages or more than %2$s pages.","uploadModal.community.buttonTitle":"Upload file","uploadModal.community.capReached.features1":"AI Studio not available","uploadModal.community.capReached.features2":"Earn points by helping others","uploadModal.community.features1":"Earn points by helping others","uploadModal.community.features2":"Use AI Studio Basic for free","uploadModal.community.features3":"Earn points by helping others","uploadModal.community.featuresRemaining":"Use AI Studio for free (%1$s %2$s)","uploadModal.community.points":"points","uploadModal.community.premium.features1":"Process your content with AI","uploadModal.community.title":"Upload to the community","uploadModal.community.usePlural":"remaining uses","uploadModal.community.useSingular":"remaining use","uploadModal.dropArea":"Drag and drop files","uploadModal.maxSizeTip":"PDFs of max %1$s MB that have between %2$s and %3$s pages","uploadModal.private.features1":"Upload documents that are only visible to you","uploadModal.private.features2":"Use AI Studio without limits","uploadModal.private.features3":"Included in your subscription","uploadModal.private.title":"Upload privately","uploadModal.selectButton":"Choose file","uploadModal.sellerBox.becomeSeller":"\u003cu\u003e**Become a seller!**\u003c/u\u003e","uploadModal.sellerBox.description":"Do you want to make money with your notes?","uploadModal.sellerBox.isSeller":"\u003cu\u003e**Sell Documents**\u003c/u\u003e","uploadModal.title":"We're uploading the document","uploadModal.uploadArea":"Upload your file","uploadModal.uploadTypeLimitations":"Photos and scans are not compatible","uploadModal.wrongFile":"Wrong type of file","word.common.documenti":"Documents","word.common.domande":"questions","word.common.maturita":"maturity","word.common.quiz":"Quiz","word.common.test":"test","word.common.universita":"University","word.common.veditutte":"View all","word.common.veditutti":"View all","word.common.video":"Video Courses"}},"trace":{"page_type":"document.view","navigation_language":"en"},"locale":"en","pointsRules":{"document_upload":20,"document_review":5,"university_review":5,"professor_review":5,"user_signup":10,"user_profile_b2b":10,"answer":5,"answer_best":10,"document_prime_promoted":100,"document_upload_ai_slave":5},"enabledModules":{"isPremiumEnabled":true,"isStoreEnabled":true,"isQuizEnabled":false,"isVideoEnabled":false,"isQuestionsEnabled":true,"isOnboardingB2bEnabled":true,"isProfessorsEnabled":true,"isProfessorReviewsEnabled":false,"isFastCheckoutEnabled":true,"isBlogEnabled":true,"isAiEnabled":true,"isLeanFreeToStoreEnabled":true},"options":{"headerStrategy":"NOT_ANIMATION_NOT_ALERTS","showFooter":true,"showHeader":true},"translationsUserLang":null,"trackParams":true,"_sentryTraceData":"7760ca4e61b34a8d846d2acd82f259fe-8e9293b7975d6e14-0","_sentryBaggage":"sentry-environment=production,sentry-release=4.10.13,sentry-public_key=e22fc7b9af4a4f95b7a7c142e6fe1e5f,sentry-trace_id=7760ca4e61b34a8d846d2acd82f259fe,sentry-sample_rate=0.01,sentry-transaction=%2Fdocs,sentry-sampled=false","doc":{"id":6743221,"lang":"en","slug":"lecture-slides-on-performance-evaluation-cs-519","userId":25639075,"title":"Understanding Precision and Recall in Information Retrieval Systems","description":"An overview of precision and recall, two essential metrics used to evaluate the performance of information retrieval systems. How precision and recall are calculated, their significance, and their relationship. It also discusses the importance of these metrics in the context of user experience and system usability.","downloads":0,"reviewsCount":0,"averageVotes":"0.0","isStore":false,"isPrime":false,"isPrimeEnabled":false,"isTop":false,"isActive":true,"isReviewed":true,"isHighschool":false,"isUniversity":true,"isSitePatatabrava":false,"isSiteEbah":false,"createdAt":{"timestamp":1251662400,"date":"2009-08-30","time":"22:00:00"},"year":2009,"favouritesCount":0,"points":20,"pointsIncrease":true,"lastPriceDown":null,"thumbnail":"https://static.docsity.com/media/avatar/documents/2009/08/30/00ff1afc36c67007ebb3ac3092be70ba.jpeg","isUnlocked":false,"file":{"pages":11,"fileExt":"pdf","source":"file","isUnconverted":false,"id":4269262,"firstPageUrl":"https://static.docsity.com/documents_first_pages/2009/08/31/045402c19ac2baf27336a5c01fe1c4d7.png","policyDate":{"timestamp":1783109601,"date":"2026-07-03","time":"22:13:21"},"htmlCssKey":"documents_html/css/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/00ff1afc36c67007ebb3ac3092be70ba.css","htmlStructureKey":"documents_html/pages/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/00ff1afc36c67007ebb3ac3092be70ba.json","textContentKey":"documents_text_html/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba.html"},"country":{"lang":"en"},"typology":{"id":6,"name":"Study notes","slug":"study-notes"},"subject":{"id":352,"name":"Computer Science","slug":"computer-science-2"},"exam":{"id":58588,"name":"Computer Science","slug":"computer-science-osu"},"university":{"id":2051,"lang":"en","slug":"oregon-state-university-or","name":"Oregon State University (OSU)","country":{"code":"us"}},"professor":null,"optimization":{"label":"opt_non_performing_share_documents_priority_low","createdAt":{"timestamp":1710152484,"date":"2024-03-11","time":"11:21:24"}},"user":{"id":25639075,"username":"koofers-user-2vp","avatar":"https://static.docsity.com/media/avatar/users/default/avatar_02.png","url":"https://www.docsity.com/en/users/profile/koofers-user-2vp/home/","isActive":true,"country":{"code":"us"},"stat":{"receivedReviewsCount":0,"uploadedDocumentsCount":10,"receivedReviewsAverageVotes":0},"seller":null}},"documentCssUrl":"https://static.docsity.com/documents_html/css/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/00ff1afc36c67007ebb3ac3092be70ba.css","documentBgPolicy":"?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9zdGF0aWMuZG9jc2l0eS5jb20vZG9jdW1lbnRzX2h0bWwvaW1hZ2VzLzIwMDkvMDgvMzEvMDBmZjFhZmMzNmM2NzAwN2ViYjNhYzMwOTJiZTcwYmEvKi5wbmciLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3ODMxMDk2MDF9fX1dfQ__\u0026Key-Pair-Id=K2FDQI4R6KV6U\u0026Signature=FtoVpSJZr3Hflp1hnf2dkL8KJbEY2OIiWyIiIynrcdxbbVY6xI9szMThpKN7yvL~ycovq2GRhZ4OlVjnRXr~UVTa-TkTNJww1pcDHe2bGb3YCyYiEvzBspFDRvtKsSrKJaonVTXMcFHCS6cuGaNFoQiTiA3D55lc~ZUuaWpBqW47K~YHIM6OlI4ecoZjRiSSv9wHiXH9IX-xbd11Alel9pkyvKbolQGlaaOwjDqOo30whxXunKYYg5Pu9eSf-dwaZM7xulrY5sMJJ9V~QAxr861d3cglw7tnPOualr8rPP1rjEWKnQTNPG2F5iafZRkKOmdpi0d~IHmjqviIbTdQ8A__","documentContent":[{"html":"\u003cdiv class=\"pc pc1 w0 h0\"\u003e\u003cimg class=\"bi x0 y0 w1 h1\" alt=\"bg1\" src=\"https://static.docsity.com/documents_html/images/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/bg1.png?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9zdGF0aWMuZG9jc2l0eS5jb20vZG9jdW1lbnRzX2h0bWwvaW1hZ2VzLzIwMDkvMDgvMzEvMDBmZjFhZmMzNmM2NzAwN2ViYjNhYzMwOTJiZTcwYmEvKi5wbmciLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE3ODMxMDk2MDF9fX1dfQ__\u0026Key-Pair-Id=K2FDQI4R6KV6U\u0026Signature=FtoVpSJZr3Hflp1hnf2dkL8KJbEY2OIiWyIiIynrcdxbbVY6xI9szMThpKN7yvL~ycovq2GRhZ4OlVjnRXr~UVTa-TkTNJww1pcDHe2bGb3YCyYiEvzBspFDRvtKsSrKJaonVTXMcFHCS6cuGaNFoQiTiA3D55lc~ZUuaWpBqW47K~YHIM6OlI4ecoZjRiSSv9wHiXH9IX-xbd11Alel9pkyvKbolQGlaaOwjDqOo30whxXunKYYg5Pu9eSf-dwaZM7xulrY5sMJJ9V~QAxr861d3cglw7tnPOualr8rPP1rjEWKnQTNPG2F5iafZRkKOmdpi0d~IHmjqviIbTdQ8A__\" fetchpriority=\"high\" decoding=\"auto\"\u003e\u003cdiv class=\"t m0 x1 h2 y1 ff1 fs0 fc0 sc0 lsc\"\u003e1\u003c/div\u003e\u003cdiv class=\"t m0 x2 h3 y2 ff2 fs1 fc0 sc0 lsd ws6\"\u003ePerformance \u003c/div\u003e\u003cdiv class=\"t m0 x2 h3 y3 ff2 fs1 fc0 sc0 ls0 ws0\"\u003eEvaluation\u003c/div\u003e\u003cdiv class=\"t m0 x2 h4 y4 ff3 fs2 fc0 sc0 lse ws7\"\u003e•\u003cspan class=\"_ _0\"\u003e \u003c/span\u003eHow do you evaluate the \u003c/div\u003e\u003cdiv class=\"t m0 x3 h4 y5 ff3 fs2 fc0 sc0 lsf ws8\"\u003eperformance of an information \u003c/div\u003e\u003cdiv class=\"t m0 x3 h4 y6 ff3 fs2 fc0 sc0 ls10 ws9\"\u003eretrieval system? \u003c/div\u003e\u003cdiv class=\"t m0 x4 h5 y7 ff3 fs3 fc0 sc0 ls11 wsa\"\u003e–\u003cspan class=\"_\"\u003e \u003c/span\u003eOr compare two different \u003c/div\u003e\u003cdiv class=\"t m0 x5 h5 y8 ff3 fs3 fc0 sc0 ls12 ws6\"\u003esystems? \u003c/div\u003e\u003cdiv class=\"t m0 x6 h3 y9 ff2 fs1 fc0 sc0 ls1 wsb\"\u003eCleverdon –\u003cspan class=\"_\"\u003e \u003c/span\u003eThe \u003c/div\u003e\u003cdiv class=\"t m0 x6 h3 ya ff2 fs1 fc0 sc0 ls13 ws1\"\u003eCranfield\u003cspan class=\"_\"\u003e \u003c/span\u003eExperiments\u003c/div\u003e\u003cdiv class=\"t m0 x6 h3 yb ff2 fs1 fc0 sc0 ls14\"\u003e1950s/1960s\u003c/div\u003e\u003cdiv class=\"t m0 x6 h6 yc ff3 fs4 fc0 sc0 ls2\"\u003e•\u003cspan class=\"ff4 ls15 wsc\"\u003eTime Lag\u003c/span\u003e\u003cspan class=\"ls3 wsd\"\u003e. The interval between the \u003c/span\u003e\u003c/div\u003e\u003cdiv class=\"t m0 x7 h7 yd ff3 fs4 fc0 sc0 ls4 wse\"\u003edemand being made and the answer \u003c/div\u003e\u003cdiv class=\"t m0 x7 h7 ye ff3 fs4 fc0 sc0 ls16 wsf\"\u003ebeing given.\u003c/div\u003e\u003cdiv class=\"t m0 x6 h6 yf ff3 fs4 fc0 sc0 ls2\"\u003e•\u003cspan class=\"ff4 ls6 ws2\"\u003ePresentation\u003c/span\u003e\u003cspan class=\"ls17 ws10\"\u003e. The physical form of the \u003c/span\u003e\u003c/div\u003e\u003cdiv class=\"t m0 x7 h7 y10 ff3 fs4 fc0 sc0 ls17 ws3\"\u003eoutput.\u003c/div\u003e\u003cdiv class=\"t m0 x6 h6 y11 ff3 fs4 fc0 sc0 ls2\"\u003e•\u003cspan class=\"ff4 ls18 ws11\"\u003eUser effort\u003c/span\u003e\u003cspan class=\"ls19 ws12\"\u003e. The effort, intellectual or \u003c/span\u003e\u003c/div\u003e\u003cdiv class=\"t m0 x7 h7 y12 ff3 fs4 fc0 sc0 ls1a ws13\"\u003ephysical, demanded of the user.\u003c/div\u003e\u003cdiv class=\"t m0 x6 h6 y13 ff3 fs4 fc0 sc0 ls2\"\u003e•\u003cspan class=\"ff4 ls5 ws4\"\u003eRecall\u003c/span\u003e\u003cspan class=\"ls1b ws14\"\u003e. The ability of the system to \u003c/span\u003e\u003c/div\u003e\u003cdiv class=\"t m0 x7 h7 y14 ff3 fs4 fc0 sc0 ls1c wse\"\u003epresent all relevant documents.\u003c/div\u003e\u003cdiv class=\"t m0 x6 h6 y15 ff3 fs4 fc0 sc0 ls2\"\u003e•\u003cspan class=\"ff4 lsf ws5\"\u003ePrecision\u003c/span\u003e\u003cspan class=\"ls19 ws15\"\u003e. The ability of the system to \u003c/span\u003e\u003c/div\u003e\u003cdiv class=\"t m0 x7 h7 y16 ff3 fs4 fc0 sc0 ls6 ws16\"\u003ewithhold non-relevant documents.\u003c/div\u003e\u003cdiv class=\"t m0 x2 h3 y17 ff2 fs1 fc0 sc0 ls0 ws17\"\u003eCleverdon says \u003c/div\u003e\u003cdiv class=\"t m0 x2 h3 y18 ff2 fs1 fc0 sc0 ls7 ws18\"\u003ePrecision \u0026amp; Recall\u003c/div\u003e\u003cdiv class=\"t m0 x2 h4 y19 ff3 fs2 fc0 sc0 ls1d ws19\"\u003e•\u003cspan class=\"_ _0\"\u003e \u003c/span\u003eMeasure ability to find the \u003c/div\u003e\u003cdiv class=\"t m0 x3 h4 y1a ff3 fs2 fc0 sc0 ls8 ws1a\"\u003erelevant information\u003c/div\u003e\u003cdiv class=\"t m0 x2 h4 y1b ff3 fs2 fc0 sc0 ls9 ws1b\"\u003e•\u003cspan class=\"_ _0\"\u003e \u003c/span\u003eIf a system can’t identify \u003c/div\u003e\u003cdiv class=\"t m0 x3 h4 y1c ff3 fs2 fc0 sc0 lsc ws1c\"\u003erelevant information, what use \u003c/div\u003e\u003cdiv class=\"t m0 x3 h4 y1d ff3 fs2 fc0 sc0 ls1b ws1d\"\u003eis it? \u003c/div\u003e\u003cdiv class=\"t m0 x6 h8 y1e ff2 fs5 fc0 sc0 ls1e ws1e\"\u003eWhy not the others? \u003c/div\u003e\u003cdiv class=\"t m0 x6 h5 y1f ff3 fs3 fc0 sc0 ls11 wsa\"\u003e•\u003cspan class=\"_ _0\"\u003e \u003c/span\u003eAccording to Cleverdon:\u003c/div\u003e\u003cdiv class=\"t m0 x8 h7 y20 ff3 fs4 fc0 sc0 ls1f ws1f\"\u003e–\u003cspan class=\"_\"\u003e \u003c/span\u003eTime lag \u003c/div\u003e\u003cdiv class=\"t m0 x9 h9 y21 ff3 fs6 fc0 sc0 ls20 ws20\"\u003e•\u003cspan class=\"_ _1\"\u003e \u003c/span\u003ea function of hardware\u003c/div\u003e\u003cdiv class=\"t m0 x8 h7 y22 ff3 fs4 fc0 sc0 ls21 ws6\"\u003e–\u003cspan class=\"_\"\u003e \u003c/span\u003ePresent\u003cspan class=\"_\"\u003e \u003c/span\u003eation \u003c/div\u003e\u003cdiv class=\"t m0 x9 h9 y23 ff3 fs6 fc0 sc0 ls22 ws21\"\u003e•\u003cspan class=\"_ _1\"\u003e \u003c/span\u003esuccessful if the user can read and \u003c/div\u003e\u003cdiv class=\"t m0 xa h9 y24 ff3 fs6 fc0 sc0 ls23 ws22\"\u003eunderstand the list of references returned\u003c/div\u003e\u003cdiv class=\"t m0 x8 h7 y25 ff3 fs4 fc0 sc0 ls18 ws11\"\u003e–\u003cspan class=\"_\"\u003e \u003c/span\u003eUser effort \u003c/div\u003e\u003cdiv class=\"t m0 x9 h9 y26 ff3 fs6 fc0 sc0 ls24 ws23\"\u003e•\u003cspan class=\"_ _1\"\u003e \u003c/span\u003ecan be measured with a straightforward \u003c/div\u003e\u003cdiv class=\"t m0 xa h9 y27 ff3 fs6 fc0 sc0 ls24 ws24\"\u003eexamination of a small number of cases. \u003c/div\u003e\u003cdiv class=\"t m0 x2 h8 y28 ff2 fs5 fc0 sc0 ls25 ws25\"\u003eIn Reality\u003c/div\u003e\u003cdiv class=\"t m0 x2 h4 y29 ff3 fs2 fc0 sc0 ls10 ws9\"\u003e•\u003cspan class=\"_ _0\"\u003e \u003c/span\u003eNeed to consider the user task \u003c/div\u003e\u003cdiv class=\"t m0 x3 h4 y2a ff3 fs2 fc0 sc0 ls9\"\u003ecarefully\u003c/div\u003e\u003cdiv class=\"t m0 x4 h5 y2b ff3 fs3 fc0 sc0 ls11 ws26\"\u003e–\u003cspan class=\"_\"\u003e \u003c/span\u003eCleverdon was focusing on \u003c/div\u003e\u003cdiv class=\"t m0 x5 h5 y2c ff3 fs3 fc0 sc0 ls11 wsa\"\u003ebatch interfaces\u003c/div\u003e\u003cdiv class=\"t m0 x4 h5 y2d ff3 fs3 fc0 sc0 ls11 wsa\"\u003e–\u003cspan class=\"_\"\u003e \u003c/span\u003eInteractive browsing interfaces \u003c/div\u003e\u003cdiv class=\"t m0 x5 h5 y2e ff3 fs3 fc0 sc0 ls12 ws27\"\u003every significant (Turpin \u0026amp; Hersh)\u003c/div\u003e\u003cdiv class=\"t m0 x2 h4 y2f ff3 fs2 fc0 sc0 lsa ws28\"\u003e•\u003cspan class=\"_ _0\"\u003e \u003c/span\u003eInteractive systems\u003c/div\u003e\u003cdiv class=\"t m0 x4 h5 y30 ff3 fs3 fc0 sc0 ls11 wsa\"\u003e–\u003cspan class=\"_\"\u003e \u003c/span\u003eUser effort \u0026amp; presentation very \u003c/div\u003e\u003cdiv class=\"t m0 x5 h5 y31 ff3 fs3 fc0 sc0 ls11\"\u003eimportant\u003c/div\u003e\u003cdiv class=\"t m0 x6 h8 y28 ff2 fs5 fc0 sc0 ls26 ws29\"\u003eIn Spite of That\u003c/div\u003e\u003cdiv class=\"t m0 x6 h4 y32 ff3 fs2 fc0 sc0 lsc ws1c\"\u003e•\u003cspan class=\"_ _0\"\u003e \u003c/span\u003ePrecision \u0026amp; Recall \u003c/div\u003e\u003cdiv class=\"t m0 x8 h5 y33 ff3 fs3 fc0 sc0 ls27 ws2a\"\u003e–\u003cspan class=\"_\"\u003e \u003c/span\u003eextensively evaluated\u003c/div\u003e\u003cdiv class=\"t m0 x6 h4 y34 ff3 fs2 fc0 sc0 lsb ws6\"\u003e•\u003cspan class=\"_ _0\"\u003e \u003c/span\u003eUsability \u003c/div\u003e\u003cdiv class=\"t m0 x8 h5 y35 ff3 fs3 fc0 sc0 ls12 wsa\"\u003e–\u003cspan class=\"_\"\u003e \u003c/span\u003enot so much\u003c/div\u003e\u003c/div\u003e","isFetched":true,"page":"documents_html/pages/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/6545a4cc68f091d7d779e76cf7ae7a59.html","pageNumber":1,"type":"HTML"},{"html":"","isFetched":false,"page":"https://static.docsity.com/documents_html/pages/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/3248099a4d038c13382ebfac7a93b803.html?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9zdGF0aWMuZG9jc2l0eS5jb20vZG9jdW1lbnRzX2h0bWwvcGFnZXMvMjAwOS8wOC8zMS8wMGZmMWFmYzM2YzY3MDA3ZWJiM2FjMzA5MmJlNzBiYS8zMjQ4MDk5YTRkMDM4YzEzMzgyZWJmYWM3YTkzYjgwMy5odG1sIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNzgzMTA5NjAxfX19XX0_\u0026Key-Pair-Id=K2FDQI4R6KV6U\u0026Signature=DwL998bELLmft9-CffXT2156J8fW9QUBA3GlGc9bw5pm3lCEX7YVLdm01jYALjhHVSlyCSk9NVv93NzCc4yejTpcbMhlA8nv9hhbJjZUjYs8SYPDDBuir1n4jeKrce1-x~IWzzSuE~Wnpramc~UnhrpBY36s86yW4dI3~My5jjW2LBrMjKVxTGpIAAPzxj2AONtjLMnCIoftqItCXHyBVIZWeQ5do~Pv~4RoPEbFR9gNAsSk0IlicDvSn7lmOhDHqqblHmDioaWwv0gIlaYFAnQvbtShKO91iSaehcgjX93g-bzCvr9~tX4seoPOXupj5GubIM05I6EK1DvEXSMLsw__","pageNumber":2,"type":"HTML"},{"html":"","isFetched":true,"page":"https://static.docsity.com/documents_html/blur/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/e10118fa088ec5978e16bcf9e3eb2758.webp","pageNumber":3,"type":"IMAGE"},{"html":"","isFetched":true,"page":"https://static.docsity.com/documents_html/blur/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/211f41b30769d8cfd3388aa4a8fc3347.webp","pageNumber":4,"type":"IMAGE"},{"html":"","isFetched":true,"page":"https://static.docsity.com/documents_html/blur/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/275849aa1365165e43176b37544830fa.webp","pageNumber":5,"type":"IMAGE"},{"html":"","isFetched":false,"page":"https://static.docsity.com/documents_html/pages/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/4f177a1a06525db2e163930cb3fc4121.html?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9zdGF0aWMuZG9jc2l0eS5jb20vZG9jdW1lbnRzX2h0bWwvcGFnZXMvMjAwOS8wOC8zMS8wMGZmMWFmYzM2YzY3MDA3ZWJiM2FjMzA5MmJlNzBiYS80ZjE3N2ExYTA2NTI1ZGIyZTE2MzkzMGNiM2ZjNDEyMS5odG1sIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNzgzMTA5NjAxfX19XX0_\u0026Key-Pair-Id=K2FDQI4R6KV6U\u0026Signature=DQd53aRZQ4J1H0hrpQxs6NKwLIgOsRJPtmdglhWBwhNZ2Ffid226jPKQ4pcQiJbdaKQntcRisbpR9LppBYcLrgmOfbqdsCwR3nk7TEjSp0GyDLKQwKsLSnokEZDcklGpy-UzzauOUHo7qlnurFoEvXIL3mL1TbwsuqT9Xm2Fx-ZJo0jtudNQK-p0cIu~4~WFKYjXnoG3wS9D88C8qwxpadAPX~9xyMrn3fw8g2og4sHR00rOziI4nw6B9gLQZwNCWEVwQ56vCBykW6nECXim-DZ7MP2zfL68-B3j01uG2nepa-XfMK3ui6JQTX1MsiX35uahNydI0-mnE9yK6EsUSA__","pageNumber":6,"type":"HTML"},{"html":"","isFetched":false,"page":"https://static.docsity.com/documents_html/pages/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/19811673bc69e9faf37bf320f263c73d.html?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9zdGF0aWMuZG9jc2l0eS5jb20vZG9jdW1lbnRzX2h0bWwvcGFnZXMvMjAwOS8wOC8zMS8wMGZmMWFmYzM2YzY3MDA3ZWJiM2FjMzA5MmJlNzBiYS8xOTgxMTY3M2JjNjllOWZhZjM3YmYzMjBmMjYzYzczZC5odG1sIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNzgzMTA5NjAxfX19XX0_\u0026Key-Pair-Id=K2FDQI4R6KV6U\u0026Signature=QoDIjtg6ZybTvaC5ySsckaJS-7HNXrfWrhunniQo~ycar60-Y~vjcWsMfJuxcM9z5hCCt5~as-~yB2wO7RR0k8SYB6cTJ1XnFfcO0nLOXM3fratie6eKiTjuOyWL-zgK90~L6B5-9IiP78WP8A4oAHsQyiJ~BrXrVQoYQ2g-MxSgDaAZPzaTf~-yfXOl8j0ChbBZwcFIhd6jAEMa80qKo4yntcrVpPIwkbHPvet5Qu56GchonjH6vhieBy7JzGC0Msa0Kr4RZNs0gDAZievpO6cFpDdy-HMj7veyvzK~hvtqCwKMfQl-ZEFRjnC5G5PuEv5OmwGoI8DEiMc3JJWN6w__","pageNumber":7,"type":"HTML"},{"html":"","isFetched":true,"page":"https://static.docsity.com/documents_html/blur/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/9ca4a90040cb5c79ed899e3746bcb4b4.webp","pageNumber":8,"type":"IMAGE"},{"html":"","isFetched":true,"page":"https://static.docsity.com/documents_html/blur/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/08723948469814681e358ffcc1368ac6.webp","pageNumber":9,"type":"IMAGE"},{"html":"","isFetched":true,"page":"https://static.docsity.com/documents_html/blur/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/68075bddb31f05df04f2910fe40eebfd.webp","pageNumber":10,"type":"IMAGE"},{"html":"","isFetched":false,"page":"https://static.docsity.com/documents_html/pages/2009/08/31/00ff1afc36c67007ebb3ac3092be70ba/dbed7ad9293540345424970473f66f51.html?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9zdGF0aWMuZG9jc2l0eS5jb20vZG9jdW1lbnRzX2h0bWwvcGFnZXMvMjAwOS8wOC8zMS8wMGZmMWFmYzM2YzY3MDA3ZWJiM2FjMzA5MmJlNzBiYS9kYmVkN2FkOTI5MzU0MDM0NTQyNDk3MDQ3M2Y2NmY1MS5odG1sIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNzgzMTA5NjAxfX19XX0_\u0026Key-Pair-Id=K2FDQI4R6KV6U\u0026Signature=qJjMcONs3PwwxzOqx7au48E0nN5YkMTSPvKxn2yJJkvw9RU7VCoH5Yw07~3OZgC9IPHtaOQ8p5JOcqrNDlOErpa5Xz1DQIXhlk~DX5zZtZv0JzkAW0PoirjmhrEdjG1ycWaoFcVQkdXOXgnv-hFdVo~PSOrK8jzlKadv0LoXX7Mhk4GQu96IgJDidqmwhBv1XEYLvWHYUB4xH7YqJcPG4I0g~~k3Sj9VgyOIfzHEAjUm0AX2WCYSGnxFgVMHEJGS8JeHL~iawbu9demDAA-wo5lEnd0vU2~86YlEWxsootAG6U1DcCQZMnnR8mvfjDQIaGK2Dr1RDAVp96bQYPRNMA__","pageNumber":11,"type":"HTML"}],"documentStructure":{"maxWidth":1240,"maxHeight":1604,"pages":[{"className":"pf w0 h0","width":"1240","height":"1604.705882"},{"className":"pf w0 h0","width":"1240","height":"1604.705882"},{"className":"pf w0 h0","width":"1240","height":"1604.705882"},{"className":"pf w0 h0","width":"1240","height":"1604.705882"},{"className":"pf w0 h0","width":"1240","height":"1604.705882"},{"className":"pf w0 h0","width":"1240","height":"1604.705882"},{"className":"pf w0 h0","width":"1240","height":"1604.705882"},{"className":"pf w0 h0","width":"1240","height":"1604.705882"},{"className":"pf w0 h0","width":"1240","height":"1604.705882"},{"className":"pf w0 h0","width":"1240","height":"1604.705882"},{"className":"pf w0 h0","width":"1240","height":"1604.705882"}],"sizes":[{"width":"1240","height":"1604.705882","numberOfPages":11}]},"documentText":"\u003ch2\u003ePerformance\u003c/h2\u003e \u003ch2\u003eEvaluation\u003c/h2\u003e \u003cul\u003e \u003cli\u003eHow do you evaluate the\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003eperformance of an information\u003c/h3\u003e \u003ch3\u003eretrieval system?\u003c/h3\u003e \u003cul\u003e \u003cli\u003eOr compare two different\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003esystems?\u003c/h4\u003e \u003ch2\u003eCleverdon – The\u003c/h2\u003e \u003ch2\u003eCranfield Experiments\u003c/h2\u003e \u003ch2\u003e1950s/1960s\u003c/h2\u003e \u003cul\u003e \u003cli\u003e\u003cstrong\u003eTime Lag\u003c/strong\u003e. The interval between the demand being made and the answer being given.\u003c/li\u003e \u003cli\u003e\u003cstrong\u003ePresentation\u003c/strong\u003e. The physical form of the output.\u003c/li\u003e \u003cli\u003e\u003cstrong\u003eUser effort\u003c/strong\u003e. The effort, intellectual or physical, demanded of the user.\u003c/li\u003e \u003cli\u003e\u003cstrong\u003eRecall\u003c/strong\u003e. The ability of the system to present all relevant documents.\u003c/li\u003e \u003cli\u003e\u003cstrong\u003ePrecision\u003c/strong\u003e. The ability of the system to withhold non-relevant documents.\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eCleverdon says\u003c/h2\u003e \u003ch2\u003ePrecision \u0026amp; Recall\u003c/h2\u003e \u003cul\u003e \u003cli\u003eMeasure ability to find the\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003erelevant information\u003c/h3\u003e \u003cul\u003e \u003cli\u003eIf a system can’t identify\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003erelevant information, what use\u003c/h3\u003e \u003ch3\u003eis it?\u003c/h3\u003e \u003cp\u003e\u003cstrong\u003eWhy not the others?\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eAccording to Cleverdon: \u003cul\u003e \u003cli\u003eTime lag \u003cul\u003e \u003cli\u003ea function of hardware\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003ePresentation \u003cul\u003e \u003cli\u003esuccessful if the user can read and understand the list of references returned\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eUser effort \u003cul\u003e \u003cli\u003ecan be measured with a straightforward examination of a small number of cases.\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eIn Reality\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eNeed to consider the user task\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003ecarefully\u003c/h3\u003e \u003cul\u003e \u003cli\u003eCleverdon was focusing on\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003ebatch interfaces\u003c/h4\u003e \u003cul\u003e \u003cli\u003eInteractive browsing interfaces\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003every significant (Turpin \u0026amp; Hersh)\u003c/h4\u003e \u003cul\u003e \u003cli\u003eInteractive systems \u003cul\u003e \u003cli\u003eUser effort \u0026amp; presentation very\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003eimportant\u003c/h4\u003e \u003cp\u003e\u003cstrong\u003eIn Spite of That\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003ePrecision \u0026amp; Recall \u003cul\u003e \u003cli\u003eextensively evaluated\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eUsability \u003cul\u003e \u003cli\u003enot so much\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eWhy Not Usability\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eUsability requires a user-study \u003cul\u003e \u003cli\u003eEvery new feature needs a new study (expensive)\u003c/li\u003e \u003cli\u003eHigh variance – many confounding factors\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eOffline analysis of accuracy \u003cul\u003e \u003cli\u003eOnce a dataset is found \u003cul\u003e \u003cli\u003eEasy to control factors\u003c/li\u003e \u003cli\u003eRepeatable\u003c/li\u003e \u003cli\u003eAutomatic\u003c/li\u003e \u003cli\u003eFree\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eIf the system isn’t accurate, it isn’t going to be usable\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003ePrecision \u0026amp; Recall\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eFor a query, documents are\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003erelevant or not\u003c/h4\u003e \u003cul\u003e \u003cli\u003eThe Relevant Set\u003c/li\u003e \u003cli\u003eThe Non-Relevant Set\u003c/li\u003e \u003cli\u003eA system retrieves a set of\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003edocuments\u003c/h4\u003e \u003cul\u003e \u003cli\u003eThe Selected Set\u003c/li\u003e \u003cli\u003eThe Non-selected Set\u003c/li\u003e \u003cli\u003eRelevance is binary!\u003c/li\u003e \u003cli\u003eEither a document is valuable or not, nothing in between\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003ePrecision and Recall\u003c/strong\u003e\u003c/p\u003e \u003ch5\u003eSelected Not Selected Total\u003c/h5\u003e \u003ch5\u003eRelevant Nrs N rn N r\u003c/h5\u003e \u003ch5\u003eIrrelevant N is N in Ni\u003c/h5\u003e \u003ch5\u003eTotal N s N n N\u003c/h5\u003e \u003cp\u003e\u003cstrong\u003ePrecision\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003ePercentage of documents selected\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003ethat are relevant\u003c/h4\u003e \u003cul\u003e \u003cli\u003eProbability that a selected\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003edocument is relevant\u003c/h4\u003e \u003cul\u003e \u003cli\u003eHow well does the system filter out\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003enon-relevant stuff\u003c/h4\u003e \u003cp\u003e\u003cem\u003es\u003c/em\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003ers\u003c/em\u003e\u003c/p\u003e \u003ch2\u003eN\u003c/h2\u003e \u003ch2\u003eN\u003c/h2\u003e \u003ch2\u003eP =\u003c/h2\u003e \u003cp\u003e\u003cstrong\u003eRecall\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003ePercentage of all relevant\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003edocuments selected\u003c/h4\u003e \u003cul\u003e \u003cli\u003eProbability that a given relevant\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003edocument will be retrieved\u003c/h4\u003e \u003cul\u003e \u003cli\u003eHow complete are the results?\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cem\u003er\u003c/em\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003er s\u003c/em\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003eN\u003c/em\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003eN R\u003c/em\u003e =\u003c/p\u003e \u003ch2\u003eHow Big is the Selected\u003c/h2\u003e \u003ch2\u003eSet?\u003c/h2\u003e \u003cul\u003e \u003cli\u003e1? 5? 10? 100?\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003ePrecision vs. Recall\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eWhich one is more important?\u003c/li\u003e \u003cli\u003eDepends on the task! \u003cul\u003e \u003cli\u003eGeneric web search engine \u003cul\u003e \u003cli\u003ePrecision!\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eIndex of court cases \u003cul\u003e \u003cli\u003eNeed all legal precedents!\u003c/li\u003e \u003cli\u003eRecall!\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eInterpolation in\u003c/h2\u003e \u003ch2\u003ePrecision/Recall\u003c/h2\u003e \u003ch2\u003eMeasurement\u003c/h2\u003e \u003cul\u003e \u003cli\u003ePrecision/Recall Table or Graph\u003c/li\u003e \u003cli\u003eComputing average precision \u003cul\u003e \u003cli\u003eAt fixed levels of recall (0,10,...,90,100)\u003c/li\u003e \u003cli\u003eAcross many different queries\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eEach query has a different number of relevant documents - Some queries – can’t compute precision at the fixed level of recall - Imagine 5 relevant documents to query – can only compute precision at recall of 0, 20, 40, 60, 80, 100.\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eInterpolation Solution\u003c/h2\u003e \u003cul\u003e \u003cli\u003eIf precision is not measured at\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003eexact recall level\u003c/h4\u003e \u003cul\u003e \u003cli\u003eUse precision measured at next highest level of recall\u003c/li\u003e \u003cli\u003eFor 5 relevant document example \u003cul\u003e \u003cli\u003eFor 10% fixed recall level, use the precision measured at 20%\u003c/li\u003e \u003cli\u003eNote this means that recall at 10% and recall at 20% will be identical for that query\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003ePrecision vs. Recall\u003c/strong\u003e\u003c/p\u003e \u003cp\u003eRong Jin, Alex G. Hauptmann, Cheng Xiang Zhai. \u003cem\u003eTitle language model for information retrieval.\u003c/em\u003e 2002. Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval.\u003c/p\u003e \u003cp\u003eAverage precision across all user queries\u003c/p\u003e \u003ch2\u003eReference Data\u003c/h2\u003e \u003ch2\u003eCollections\u003c/h2\u003e \u003cul\u003e \u003cli\u003eTREC \u003cul\u003e \u003cli\u003eText Retrieval Conference\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eCACM \u003cul\u003e \u003cli\u003eCommunications of the ACM\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eISI \u003cul\u003e \u003cli\u003eInstitute for Scientific Information\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eTREC\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eMuch more than a dataset\u003c/li\u003e \u003cli\u003eText Retrieval Conference \u003cul\u003e \u003cli\u003eStarted in 1992 by Donna Harman\u003c/li\u003e \u003cli\u003eSponsored by the National Institute of Standards and Technology (US Commerce Dept.) and the Defense Advanced Research Projects Agency (DARPA)\u003c/li\u003e \u003cli\u003eHeld every year\u003c/li\u003e \u003cli\u003eOnly those who perform the TREC retrieval tasks for that year can participate\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eA TREC Conference\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eSplit into \u003cstrong\u003etracks\u003c/strong\u003e \u003cul\u003e \u003cli\u003eEach track evaluates a different user task\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003e2003 Tracks \u003cul\u003e \u003cli\u003eCross-language\u003c/li\u003e \u003cli\u003eFiltering\u003c/li\u003e \u003cli\u003eGenome\u003c/li\u003e \u003cli\u003eHigh accuracy\u003c/li\u003e \u003cli\u003eInteractive\u003c/li\u003e \u003cli\u003eNovelty\u003c/li\u003e \u003cli\u003eQuestion-Answering\u003c/li\u003e \u003cli\u003eRobust retrieval\u003c/li\u003e \u003cli\u003eDigital video\u003c/li\u003e \u003cli\u003eWeb\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eTREC Procedure\u003c/strong\u003e\u003c/p\u003e \u003cp\u003e(for the ad-hoc task)\u003c/p\u003e \u003cul\u003e \u003cli\u003eEach participant is provided \u003cul\u003e \u003cli\u003e\u003cem\u003einformation requests\u003c/em\u003e , known as topics\u003c/li\u003e \u003cli\u003eDocument collection\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eEach participant must then \u003cul\u003e \u003cli\u003eConvert information requests to queries\u003c/li\u003e \u003cli\u003eReturn ranked results\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eNIST then \u003cul\u003e \u003cli\u003eIdentifies the relevant documents (using pooling)\u003c/li\u003e \u003cli\u003eReports the precision and recall of each participatingsystem\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eParticipants \u003cul\u003e \u003cli\u003eWrite a paper describing their system and analyzing their results\u003c/li\u003e \u003cli\u003ePresent the paper at the TREC workshop\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eExample Topic\u003c/strong\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003eFrom TREC-7 ad-hoc track\u003c/em\u003e \u003ctop\u003e \u003cnum\u003e Number: 363\u003ctitle\u003e transportation tunnel disasters\u003c/p\u003e \u003cp\u003e\u003cdesc\u003e Description:What disasters have occurred in tunnels used for transportation? \u003cnarr\u003e Narrative:A relevant document identifies a disaster in a tunnel used for trains, motor vehicles, or people. Wind tunnels and tunnels used for wiring, sewage, water, oil,etc. are not relevant. The cause of the problem may be fire, earthquake, flood, or explosion and can beaccidental or planned. Documents that discuss tunnel disasters occurring during construction of a tunnelare relevant if lives were threatened.\u003c/p\u003e \u003cp\u003e\u003c/top\u003e\u003c/p\u003e \u003ch2\u003eDocuments Used in\u003c/h2\u003e \u003ch2\u003eTREC\u003c/h2\u003e \u003cul\u003e \u003cli\u003eWall Street Journal, Federal\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003eRegister, Associated Press,\u003c/h3\u003e \u003ch3\u003eDepartment of Energy\u003c/h3\u003e \u003ch3\u003eabstracts, Financial Times,\u003c/h3\u003e \u003ch3\u003eCongressional Record, Foreign\u003c/h3\u003e \u003ch3\u003eBroadcast Information Service,\u003c/h3\u003e \u003ch3\u003eLos Angeles Times, .GOV web\u003c/h3\u003e \u003ch3\u003edocuments, references from\u003c/h3\u003e \u003ch3\u003eMEDLINE\u003c/h3\u003e \u003cp\u003e\u003cstrong\u003eObtaining TREC Data\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eNot Free\u003c/li\u003e \u003cli\u003eMust sign data use agreement\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eCACM\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eArticles about computer\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003escience research and practice\u003c/h3\u003e \u003cul\u003e \u003cli\u003eIncludes metadata \u003cul\u003e \u003cli\u003eAuthors\u003c/li\u003e \u003cli\u003eDates\u003c/li\u003e \u003cli\u003eSubject Categories\u003c/li\u003e \u003cli\u003eBibliographic linkage and co-\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003ecitation\u003c/h4\u003e \u003cp\u003e\u003cstrong\u003eISI\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eDocuments about information\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003escience\u003c/h3\u003e \u003cul\u003e \u003cli\u003eMetadata \u003cul\u003e \u003cli\u003eBibliographic co-citations\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eBibliographic Citation\u003c/h2\u003e \u003ch2\u003eInformation in CACM\u003c/h2\u003e \u003cul\u003e \u003cli\u003eDirect references \u003cul\u003e \u003cli\u003eA referenced B\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eBibliographic coupling \u003cul\u003e \u003cli\u003e[a, b, n] (n documents were cited by both a and b)\u003c/li\u003e \u003cli\u003eIndicates potential similarity\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eCo-citations \u003cul\u003e \u003cli\u003e[a, b, n] (a and b appear together in a bibliography in n different documents)\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eMany Other Datasets of\u003c/h2\u003e \u003ch2\u003eInterest\u003c/h2\u003e \u003cul\u003e \u003cli\u003eMany other datasets are available\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003ebesides the ones mentioned\u003c/h4\u003e \u003cul\u003e \u003cli\u003eMost don’t have relevance data\u003c/li\u003e \u003cli\u003eOthers\u003c/li\u003e \u003cli\u003eEachMovie – collaborative filtering\u003c/li\u003e \u003cli\u003eMachine Learning Repository at U. of California – Irvine.\u003c/li\u003e \u003cli\u003eIncludes a SPAM dataset\u003c/li\u003e \u003cli\u003eMost don’t have text\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eQuery Language\u003c/h2\u003e \u003ch2\u003eFeatures\u003c/h2\u003e \u003cul\u003e \u003cli\u003eImportant to understand the\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003edifferent query capabilities of\u003c/h3\u003e \u003ch3\u003ethe popular search engines\u003c/h3\u003e \u003cp\u003e\u003cstrong\u003eQuery Elements\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eWords (index terms) \u003cul\u003e \u003cli\u003eGenerally automatically\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003eextracted\u003c/h4\u003e \u003cul\u003e \u003cli\u003eWhat defines a word? \u003cul\u003e \u003cli\u003eSeparated by spaces?\u003c/li\u003e \u003cli\u003eWhat about other characters?\u003c/li\u003e \u003cli\u003eContractions?\u003c/li\u003e \u003cli\u003eHyphens?\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eOther Query Options\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003ePhrases \u003cul\u003e \u003cli\u003e“Oh! the difference of man and man” ( \u003cem\u003eK\u003c/em\u003e \u003cem\u003eLear, Act iv, Sc.2\u003c/em\u003e )\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eProximity \u003cul\u003e \u003cli\u003e“Konstan w/2 Herlocker” (Citeseer)\u003c/li\u003e \u003cli\u003e“Destruction \u003cnear\u003e Oregon” (NSF – Verity) - \u003cstrong\u003eTsunamis Affecting the United States\u003c/strong\u003e \u003cstrong\u003eWest Coast 1812-\u003c/strong\u003e\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eConsider Indexing of\u003c/h2\u003e \u003ch2\u003eProximity...\u003c/h2\u003e \u003cul\u003e \u003cli\u003ePhrases \u003cul\u003e \u003cli\u003eCould treat each separate\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003ephrase as a keyword\u003c/h4\u003e \u003cul\u003e \u003cli\u003eCombinatorial expansion!\u003c/li\u003e \u003cli\u003eWhat about NEAR?\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eOther Query\u003c/h2\u003e \u003ch2\u003ePossibilities\u003c/h2\u003e \u003cul\u003e \u003cli\u003ePrefixes (also known as truncation) \u003cul\u003e \u003cli\u003e(Altavista) Herloc* \u003cul\u003e \u003cli\u003e\u003cstrong\u003eWelcome to Herlocher's Dipping Mustard!!\u003c/strong\u003e\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eOther string patterns \u003cul\u003e \u003cli\u003eSubstrings, suffixes, regular expressions, etc\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eAccounting for errors \u003cul\u003e \u003cli\u003eglimpse -1 unbelievable (allows one character spellingerror)\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eSoundex \u003cul\u003e \u003cli\u003e“Sounds like”\u003c/li\u003e \u003cli\u003ePopular in genealogy and the census\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eExamine the documentation for your favorite search engines\u003c/li\u003e \u003cli\u003eA nice site with a feature table is \u003cul\u003e \u003cli\u003ehttp://searchengineshowdown.com/features/\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eSome IR Challenges\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eUsers need information\u003c/li\u003e \u003cli\u003eThey often can’t describe their need perfectly - i.e. Anomalous State of Knowledge (ASK)\u003c/li\u003e \u003cli\u003eOr perhaps they can only describe it in “their” terminology - Which might be different from others\u003c/li\u003e \u003cli\u003eOr perhaps they are lazy (or time constrained) - Can’t/won’t spend the effort to design a “good” query\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eQuery Expansion!\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eCan we automatically expand\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003ethe query?\u003c/h3\u003e \u003cul\u003e \u003cli\u003eGoal: improve the\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003eeffectiveness of search by\u003c/h3\u003e \u003ch3\u003ecreating a “better” query\u003c/h3\u003e \u003cp\u003e\u003cstrong\u003eRelevance Feedback\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eUser issues short, simple query\u003c/li\u003e \u003cli\u003eFrom retrieved results \u003cul\u003e \u003cli\u003eUser indicates relevant documents\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eRelevant documents are\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003econsidered an “example” of a more\u003c/h4\u003e \u003ch4\u003edetailed query\u003c/h4\u003e \u003cul\u003e \u003cli\u003eFind documents similar to the\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003eidentified relevant documents\u003c/h4\u003e \u003ch2\u003eEvaluating Relevance\u003c/h2\u003e \u003ch2\u003eFeedback\u003c/h2\u003e \u003cul\u003e \u003cli\u003eNeed to do a user experiment \u003cul\u003e \u003cli\u003eRelevant documents selected by\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003euser depend on the original\u003c/h4\u003e \u003ch4\u003esearch results\u003c/h4\u003e \u003cul\u003e \u003cli\u003ePrecision and Recall \u003cul\u003e \u003cli\u003eSuppose you measure P\u0026amp;R of\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003ethe results from the expanded\u003c/h4\u003e \u003ch4\u003equery?\u003c/h4\u003e \u003ch2\u003ePrec. \u0026amp; Recall with\u003c/h2\u003e \u003ch2\u003eRelevance Feedback\u003c/h2\u003e \u003cul\u003e \u003cli\u003eUser provides initial query\u003c/li\u003e \u003cli\u003eSystem provides initial results (say 10)\u003c/li\u003e \u003cli\u003eUser identifies two relevant documents at ranks 8 and 10\u003c/li\u003e \u003cli\u003eSystem provides updated results based on R. F.\u003c/li\u003e \u003cli\u003eThe relevant documents that were originally at ranks 8 and 10 are now at ranks 1 and 2 - An easy cheat!\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003ePrecision and Recall \u0026amp;\u003c/h2\u003e \u003ch2\u003eRelevance Feedback\u003c/h2\u003e \u003cul\u003e \u003cli\u003eResidual collection precision\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003eand recall\u003c/h3\u003e \u003cul\u003e \u003cli\u003eCompute precision and recall\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003eafter removing all documents\u003c/h4\u003e \u003ch4\u003eused as feedback.\u003c/h4\u003e \u003cul\u003e \u003cli\u003eConsider the user interaction \u003cul\u003e \u003cli\u003eCould compare relevance\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003efeedback to manual query\u003c/h4\u003e \u003ch4\u003ereformulation\u003c/h4\u003e \u003ch2\u003eFully Automatic Query\u003c/h2\u003e \u003ch2\u003eExpansion\u003c/h2\u003e \u003cul\u003e \u003cli\u003eRelevance feedback requires user\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003einput\u003c/h4\u003e \u003cul\u003e \u003cli\u003eUser provides examples of relevant and non-relevant documents\u003c/li\u003e \u003cli\u003eSystem extracts words from those documents and adds them or removes them from the query\u003c/li\u003e \u003cli\u003eWhat can we do without user input?\u003c/li\u003e \u003cli\u003eStatistical thesaurus expansion\u003c/li\u003e \u003c/ul\u003e \u003cp\u003e\u003cstrong\u003eConsider\u003c/strong\u003e\u003c/p\u003e \u003cul\u003e \u003cli\u003eUser searches for “cat”\u003c/li\u003e \u003cli\u003eWon’t see documents that use the\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003eword “feline” instead of cat\u003c/h4\u003e \u003cul\u003e \u003cli\u003eThus terms are not independent\u003c/li\u003e \u003cli\u003eA potentially better search\u003c/li\u003e \u003cli\u003e“cat OR feline”\u003c/li\u003e \u003cli\u003eIf you had a thesaurus...\u003c/li\u003e \u003cli\u003eLook up each query term\u003c/li\u003e \u003cli\u003eAdd all the synonyms to the query\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eConstruct a Thesaurus\u003c/h2\u003e \u003cul\u003e \u003cli\u003eMay not have a useful\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003ethesaurus available\u003c/h3\u003e \u003cul\u003e \u003cli\u003eResearch has shown that\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003egeneric thesauri don’t fare well\u003c/h4\u003e \u003cul\u003e \u003cli\u003eLose too much precision\u003c/li\u003e \u003cli\u003eNeed a thesaurus specific to\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003ethe topics in your query\u003c/h3\u003e \u003cul\u003e \u003cli\u003eCan you construct one?\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eUse Document\u003c/h2\u003e \u003ch2\u003eCollection to Construct\u003c/h2\u003e \u003ch2\u003eThesaurus\u003c/h2\u003e \u003cul\u003e \u003cli\u003eAnalyze the text in the document store\u003c/li\u003e \u003cli\u003eGlobal \u003cul\u003e \u003cli\u003eBuild a thesaurus from all documents\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eLocal \u003cul\u003e \u003cli\u003eFirst retrieve documents based on initial query\u003c/li\u003e \u003cli\u003eBuild a thesaurus from those documents\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003cli\u003eExpand query using thesaurus \u003cul\u003e \u003cli\u003eRerun query\u003c/li\u003e \u003c/ul\u003e\u003c/li\u003e \u003c/ul\u003e \u003ch2\u003eCo-occurrences\u003c/h2\u003e \u003cul\u003e \u003cli\u003eIf two words frequently co-\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003eoccur\u003c/h3\u003e \u003cul\u003e \u003cli\u003eThey might be related\u003c/li\u003e \u003cli\u003eFor example\u003c/li\u003e \u003cli\u003eAny document that mentions\u003c/li\u003e \u003c/ul\u003e \u003ch4\u003e“feline” frequently will include\u003c/h4\u003e \u003ch4\u003emention “cat” frequently\u003c/h4\u003e \u003ch2\u003eAssociation Clusters\u003c/h2\u003e \u003ch5\u003ej l\u003c/h5\u003e \u003ch5\u003eu v\u003c/h5\u003e \u003ch2\u003ed D\u003c/h2\u003e \u003ch2\u003eu v k j k j\u003c/h2\u003e \u003ch2\u003ec f f\u003c/h2\u003e \u003cp\u003eCorrelation\u003c/p\u003e \u003cp\u003eDocument\u003c/p\u003e \u003cp\u003eAll Retrieved Documents\u003c/p\u003e \u003cp\u003eFrequency of k (^) u in document j\u003c/p\u003e ","relatedDocs":[{"id":6236537,"lang":"en","slug":"information-retrieval-lecture-slides-cs-6740","title":"Information Retrieval: Evaluation and Metrics","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":5101086,"lang":"en","slug":"kubat-machine-learning-1","title":"Machine Learning Performance Evaluation: Precision, Recall, Error Rate \u0026 More","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":8916304,"lang":"en","slug":"q1-1-consider-following-statements-and-choose-the-correct","title":"Machine Learning Quiz: ROC, Precision-Recall Curves, \u0026 Evaluating Imbalanced Datasets","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":6850836,"lang":"en","slug":"calculation-of-the-evaluation-matrices-homework-5-cs-410","title":"Information Retrieval Assignment: Calculating Evaluation Metrics for CS 410/510 - Prof. Da","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":6811618,"lang":"en","slug":"evaluation-and-relevance-information-retrieval-on-the-internet-cs-510","title":"Information Retrieval Evaluation: Metrics and Test Collections in CS 510 - Prof. David E. ","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":6771534,"lang":"en","slug":"information-filtering-and-retrieval-introduction-to-ir-cs-419","title":"Information Retrieval: Data vs. Information and Models for Information Retrieval","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":7452644,"lang":"en","slug":"performance-measures-for-machine-learning","title":"Machine Learning Performance: Accuracy, Lift, Precision, Recall, ROC, and ROC Area","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":6505259,"lang":"en","slug":"question-answering-information-retrieval-cmpsci-646","title":"Question Answering in Information Retrieval: From Document Retrieval to Answer Retrieval -","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":15916495,"lang":"en","slug":"precision-product-line-precision-product-line","title":"Precision Product Line----------------Precision Product Line","isStore":true,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":38687,"lang":"en","slug":"information-retrieval-tolerant-retrieval-lecture-slides-computer-science","title":"Information Retrieval Tolerant Retrieval, Lecture Slides - Computer Science","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":6752419,"lang":"en","slug":"3-questions-on-vector-space-concepts-recall-and-precision-midterm-cs-4300","title":"3 Questions on Vector Space Concepts, Recall and Precision - Midterm | CS 4300","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0},{"id":287865,"lang":"en","slug":"large-scale-instance-retrieval-introduction-to-computer-version-lecture-sli","title":"Large-Scale Image Retrieval: Visual Object Recognition and Instance Retrieval","isStore":false,"isPrime":false,"isTop":false,"averageVotes":"0.0","reviewsCount":0}],"suggestedDocs":[]},"__N_SSP":true},"page":"/docs","query":{"slug":"lecture-slides-on-performance-evaluation-cs-519","id":"6743221"},"buildId":"88ab84EjVlt2ImlLvPoah","assetPrefix":"https://assets.docsity.com/mf/docs/4.10.13","isFallback":false,"isExperimentalCompile":false,"gssp":true,"appGip":true,"locale":"en","locales":["default","it","en","es","pt","fr","pl","ru","sr","de"],"defaultLocale":"default","scriptLoader":[]}</script></body></html>
