GENIE: A Leaderboard for Human-in-the-Loop Evaluation of Text Generation

Daniel Khashabi, Gabriel Stanovsky, Jonathan Bragg, Nicholas Lourie, Jungo Kasai, Yejin Choi, Noah A. Smith, and Daniel S. Weld
preprint  2021

Tl;DR: This work introduces GENIE, an extensible human evaluation leaderboard, which brings the ease of leaderboards to text generation tasks. GENIE automatically posts leaderboard submissions to crowdsourcing platforms and presents both manual and automatic metrics on the leaderboard.

PAWLS: PDF Annotation With Labels and Structure

Mark Neumann, Zejiang Shen, and Sam Skjonsberg
preprint  2021

Tl;DR: PAWLS is a new annotation tool designed specifically for the PDF document format. PAWLS supports span-based textual annotation, N-ary relations and freeform, non-textual bounding boxes, all of which can be exported in convenient formats for training multi-modal machine learning models.

SciSight: Combining faceted navigation and research group detection for COVID-19 exploratory scientific search

Tom Hope, Jason Portenoy*, Kishore Vasan*, Jonathan Borchardt*, Eric Horvitz, Daniel S. Weld, Marti A. Hearst, and Jevin D. West
EMNLP  2020

Tl;DR: SciSight is a novel framework for exploratory search of COVID-19 research that integrates two key capabilities: first, exploring interactions between biomedical facets (e.g., proteins, genes, drugs, diseases, patient characteristics); and second, discovering groups of researchers and how they are co... nnected.

Explanation-Based Tuning of Opaque Machine Learners with Application to Paper Recommendation

Benjamin Charles Germain Lee, Kyle Lo, Doug Downey, and Daniel S. Weld
preprint  2020

Tl;DR: We developed a general approach for actionable explanations, which you can try within our Semantic Sanity prototype. User studies of the approach have shown that it leads to higher perceived user control, trust, and satisfaction.

  • s2 View and cite on Semantic Scholar
  • PDF View PDF