Benchmarks
Collection
My own benchmarks. All present in Obscure-Bench: https://obscureai.mooo.com/leaderboard • 5 items • Updated
Dataset Viewer
The dataset viewer is not available for this dataset.
Unexpected token '<', "<html>
<h"... is not valid JSON
Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.
ask-my-agent bench v2
Benchmark measuring usability of models to use with generic agentic apps (such as ask-my-agent)
Changelog
Updates in v2
- Make sure to remove/fix unsolvable/almost impossible tasks.
- Added more tasks
- (EVAL SUITE) Added number of each task, so I can see how much left.
- (EVAL SUITE) Now at end it also shows what model you used, is native or fallback and benchmark name.
Usage
Repository contains index.js. To run, you need:
- Configure
ask-my-agentwithnpx ask-my-agent config - Run
node index.js
Done!
Leaderboard
Benchmark has separate leaderboard at ObscureBench for native and fallback
About Native and Fallback
P.S. This is not about this benchmark much, but about ask-my-agent
Native tool calls are generally easier for model, because it's trained to do tasks like this. Fallback is entirely text-only talk, just like you would do in chat. No tools.
For this reason, fallback and native separated on leaderboard.
Brought to you by sapbot from Romarchive
- Downloads last month
- 114