Evaluation of Ontology-based Tools

2nd International Workshop

located at the 2nd International Semantic Web Conference ISWC 2003
20th October 2003 (Workshop day)
Sundial Resort, Sanibel Island, Florida, USA

In cooperation with the OntoWeb Special Interest Group (SIG) on Tools

Published as CEUR-WS Vol-87

Program & Experiment Objectives Topics Previous Workshops Organization Committee Program Committee Submission Dates

Not what you were looking for? Maybe you have been searching for Stanford's EON project, which creates components for medical decision support systems.

Here you will find the other EON workshops: EON2006, EON2004, and EON2002

Program & Experiment Top

The schedule, papers and slides of the workshop can be found here.

For this workshop we propose to experiment on tools interoperability. A description of the experiment proposed can be obtained here. We encourage you to send an expression of interest for participating in the experiment until May 15th to York Sure.

The experiment will be discussed at the OntoWeb 5 meeting on 19th October as part of the meeting of the Special Interest Group (SIG) on Tools.

Objectives Top

In the Evaluation of Ontology-based Tools workshop we intend to bring together researchers and practitioners from the fastly developing research areas Ontologies and Semantic Web. Currently the semantic web attracts researchers from all around the world. Numerous tools and applications of semantic web technologies are already available and the number is growing fast. However, deploying large scale ontology solutions typically involves several separate tasks and requires applying multiple tools. Therefore pragmatic issues such as interoperability are key if industry is to be encouraged to take up ontology technology rapidly. The main aim of this workshop is therefore to encourage and stimulate discussions about the evaluation of ontology-based tools.

The large visibility of the semantic web, it?s tools and applications already attract industrial partners . In particular, as tools move from academic institutions into commercial environments they have to fulfil stronger requirements and in some cases new requirements (e.g. concerning scalability and multi-user access). Different tools from different sources need to interoperate. Typically tools are not anymore standalone solutions but integrated into a framework. This framework must be open to other commercial applications and provide connectors and interfaces to industrial standards. Larger applications need also larger ontologies and therefore require substantially more performance and scalability.

A systematic evaluation of the tools might lead to a consistent level of quality and thus acceptance by industry. For the future this might lead into certification efforts for such tools.

This event is sponsored by OntoWeb

Topics of Interest Top

Topics of interest include but are not limited to:

  • Interoperability of tools (e.g. turnaround abilities)
  • Integration of tools into frameworks
  • Performance benchmarks
  • Scalability of tools
  • Certification of tools
Tools include e.g.:
  • Annotation tools
  • Inference engines
  • Ontology editors and browsers
  • Ontology servers
  • Ontology mapping tools
  • Repositories
  • Retrieval tools
  • Semantic search engines

Previous Workshops Top

The first workshop on Evaluation of Ontology-based Tools (EON2002) was celebrated in conjunction with the 13th International Conference on Knowledge Engineering and Knowledge Management (EKAW2002), in September 30th, 2002.

The workshop attracted a large number of researchers and practitioners of ontology-based tools (~25 participants), who partially contributed with their papers (7 of them were accepted) and with their descriptions of the experiment proposed (9 descriptions of different ontology editors).

Workshop Organising Committee Top

Program Committee Top

  • Dean Allemang, TopQuadrant, Inc. (US), dallemang@topquadrant.com
  • Bill Anderson, Ontology Works, Inc. (US), andersen@ontologyworks.com
  • Nathalie Aussenac-Gilles, Université P. Sabatier (FR), aussenac@irit.fr
  • Sean Bechhofer, University of Manchester (UK), seanb@cs.man.ac.uk
  • Richard Benjamins, iSOCO (ES), richard@isoco.com
  • John Davies, BT (UK), john.nj.davies@bt.com
  • Carole Goble, University of Manchester (UK), cgoble@cs.man.ac.uk
  • Asunción Gómez-Pérez, Universidad Politécnica de Madrid (ES), asun@fi.upm.es
  • Jeff Heflin, Lehigh University (US), heflin@cse.lehigh.edu
  • Atanas Kiryakov, OntoText Lab / Sirma AI, Ltd. (BG), naso@sirma.bg
  • Riichiro Mizoguchi, Osaka University (JP), miz@ei.sanken.osaka-u.ac.jp
  • Natasha F. Noy, Stanford University (US), noy@smi.stanford.edu
  • Leo Obrst, The MITRE Corporation, Information Semantics (US), lobrst@mitre.org
  • Henrik Oppermann, Ontoprise (DE), oppermann@ontoprise.de
  • Norman Sadeh, Carnegie Mellon University (US), sadeh@cs.cmu.edu
  • Heiner Stuckenschmidt, Vrije Universiteit Amsterdam (NL), heiner@cs.vu.nl
  • Rudi Studer, University of Karlsruhe (DE), studer@aifb.uni-karlsruhe.de
  • Mike Uschold, Boeing (US), michael.f.uschold@boeing.com

Submission and Proceedings Top

We invite three types of submissions for this workshop:

  • Technical papers in any of the topics of interest of the workshop (but not limited to them)
  • Short position papers in any of the topics of interest of the workshop (but not limited to them)
  • Papers describing the results of the experiment proposed

Submitted papers will be peer-reviewed and selected on the basis of these reviews. Accepted papers will be presented at the workshop. The workshop will be divided in two main parts: (1) presentations of technical and short position papers, followed by discussions, and (2) presentations of the results of the experiment proposed, also followed by discussions. In order to obtain an intensive exchange of ideas between the participants, there will be left extensive time for discussion following each presentation. A final panel will discuss the research agenda for the coming years and predicted development trends to serve industrial requirements.

We explicitly encourage people to make demos of their tools.

Format requirements for submissions of papers are:

  • Maximum 10 pages, excluding title page and bibliography for technical papers.
  • Maximum 10 pages, excluding title page and bibliography for experiment description papers. These submissions must also include additional material, such as the ontologies used for the experiment.
  • Maximum 2 pages, excluding title page and bibliography for short position papers.
  • All submissions should be made electronically if possible, by email attachment and preferably in Postscript or PDF format. Only if electronic submission is impossible should you send three hardcopies.
  • All submissions must be sent to the workshop contact, York Sure at sure@aifb.uni-karlsruhe.de

Although not required for the initial submission, we recommend to follow the format guidelines of ISWC 2003 (Springer LNCS), as this will be the required format for accepted papers (cf. http://www.springer.de/comp/lncs/authors.html). We will publish the workshop proceedings online at CEUR-WS.

Important Dates Top

  • Expression of interest for experiment: May 15th, 2003
  • Deadline paper submissions: July 14th, 2003
  • Notification of acceptance: August 25th, 2003
  • Camera ready deadline: September 22nd, 2003
  • Workshop: October 20th, 2003

Please do not hesitate to contact York Sure for any questions you have!