v1.1.0

What's new

Since v1.0.1, we haven't been very active when it comes to releases 😇 However, we worked hard on the application itself 💪

The current release 1.1.0 thus brings many significant improvements, some of which are detailed below, and many important bugfixes.

Important

We recommend that you use this version only for a fresh instance. The data generated by the older version of factgenie is most probably not compatible with the current version and vice versa.

🌈 Annotation interface

• New annotation library (#177, #192, #194, #210): We built our own underlying JS annotation library. That allowed us to introduce better visual style, better colors, "undo" functionality, erase / select mode, overlapping annotations, and more.

demo.mp4

• Refactored sliders (#216): We modified the way the sliders work. Now, they work in a more "traditional" mode, where the users can select continuous values, instead of duplicating the role of select boxes for multi-choice answering.

• One-based indexing (#193) : Human annotators now see the examples indexed from 1 instead of 0, which is more natural.

🖥️ Browse interface

• Public mode (#161): It is now possible to publish collected annotations online without providing access to other app parts.
• Outputs with no inputs (#164): It is now possible to have a dataset with no input data.
• New highlight style (#163, #177): The annotation highlights now follow the new style (see above).

🤖 LLM annotations

• New LLM providers (#189): Basing our API calls on the LiteLLM library, we now support annotations from Ollama, vLLM, OpenAI, Google AI Studio, Anthropic, and Vertex AI.

📈 Analysis

• IAA computation (#181, #207, #217, #218): We provide raw data for computing inter-annotator agreement, along with an accompanying Jupyter notebook and CLI interface. We also allow to compute IAA between individual groups in the same campaign.

🔧 Other

• Better logging (#205): The logs from factgenie are now more colorful and informative:

• Caching (#167): Factgenie is now much faster when handling larger amount of data.

Even though we will release a new version on PyPI, we still recommend installing factgenie as an editable Python project.

v1.0.1

First stable release

The first stable release of factgenie 🎉

Make sure to update to this release from any preliminary versions.