The evaluation of large information retrieval systems with application to MEDLARS

Abstract

The MEDLARS system consists of a large file of indexed references with facilities for retrieving all references indexed by a specified combination of index terms. Were the file small the retrieval performance of the system could be evaluated by as king a number of system users to inspect the entire file arid to compare the references they selected as relevant to their needs with the reference by a MEDLARS search. Since the file is large this is not possible. MEDLARS performance is therefore evaluated by measuring the change in the user's literature awareness induced by the system, and by comparing the standard system with three alternative retrieval techniques.