The story so far... How BirdNET-Pulse came to be
What's the Problem?
The free BirdNET analysis tools from Cornell University (birdnet.cornell.edu) are extremely widely used by ornithologists, wildlife organisations and individuals to process recordings from field recorders in order to automatically identify birds calling in those audio files. This process is known as "auto-classification" and is very popular as it removes the need to have trained ornithologists listen to 100s or 1,000s of hours of audio.
The BirdNET tools produce CSV files (with commas separating the individual data items) that contain species identification, how confident the AI is in the accuracy of the identification (a value from 0.0 to 1.0 representing 0% to 100% certainty) plus time of call and other metadata. The main tool used for this is a free Windows application called BirdNET-Analyzer (birdnet.cornell.edu/analyzer), though other tools based on BirdNET can also produce similar CSV files. AI tools such as these are not 100% accurate, though they are pretty good if you stick to the higher confidence values. They can, however, go off-piste and lose the plot, so there will be false-positives and some outliers (species that are unlikely given your location and time of year) may be missed. Use common sense and if in doubt, check the source audio. That's just the way things are!
The CSV files are rich in data but not great for easy analysis - one recorder's data for a single day may produce 1GB of audio files which, after processing by BirdNET, may produce CSV files containing several 1,000 possible identified calls. It's easy to see that even with just one recorder, every month you could have 100,000 or more CSV records - this problem just expands as the number of recorders increases.
Traditionally, these CSV files were imported into a spreadsheet tool, e.g. Excel or Google Sheets, and processed using macros and pivot tables to produce reports and graphs. This process requires technical knowledge, skill, is very time-consuming, manually intensive and thus error prone and doesn't scale well. Plus, we all have more important and exciting things to do with our lives.
Our Solution
We created BirdNET-Pulse as an online tool to automatically produce reports and graphs from the CSV files produced by BirdNET-Analyzer, Chirpity, BirdNET-Pi and similar BirdNET-based tools, e.g. BirdNET-PiPy. The key design objectives were to:
- produce meaningful reports/charts simply - a one-stop-shop for 90% of requirements
- be quick - most reports/charts take just a few seconds
- improve accuracy and repeatability - make the whole process deterministic
- make it scale well - it happily processes many 1,000,000s of records a day
- de-skill the whole reporting/charting process
- remove as many manual breaks (human interactions) as possible
- make generating reports/charts fun rather than a chore
Originally, this was just to solve the data processing problem we had as individuals and as a wildlife group (mardenwildlife.org.uk), however it soon became apparent that many similar (and much larger) organisations had an identical problem: How to get meaningful reports from all this data?
You can:
- create professional-quality graphs and reports
- use scientific names, common names or both
- select to mark each species in the reports with UK RSPB/BTO BoCC5 or European IUCN/BI Red List flags, or just report on the at-risk species in the selected list
- filter by date range
- filter by full or partial scientific or common name, so can select just a single species out of all that data, or a group, e.g. "buteo"
- select the minimum confidence level in the input data to use. We recommend something in the range of 75% to 80% (the default) - too low and the number of false positives increases; too high and you'll miss faint/boundary identifications
- output reports as MS Word (DOCX), Adobe (PDF) or generic spreadsheet (CSV) files
- download high resolution charts as PNG, SVG or WebP files
Wait! There's more!
We soon realised that processing the CSV files was just part of the problem - generating the CSV files was also an issue, especially for non-techies. BirdNET-Analyzer, amazing though it may be, is not especially user-friendly, particularly for those who are not experienced ornithologists - to process all the audio files is in itself a time-consuming task - there are quite a few values to set for each run as BirdNET-Analyzer doesn't (as of v2.4) have a facility to save & restore settings. Other settings require some technical nous and rely on knowledge of the hardware (PC) that the analysis is being run on; to get the fastest possible run, BirdNET-Analyzer needs to process as many files as possible at the same time, e.g. if your PC is capable of doing 20 things at the same time, the analyser should really be running 19 audio files at the same time, leaving one for things like the mouse and updating the screen etc.
Something that worked out your required settings would be a GOOD THING!
So we came up with the BirdNET-Runner program. This is a small utility that determines the optimal settings for your PC and then runs BirdNET-Analyzer in the fastest way possible for your PC.
It's simple and quick and can be installed from the Downloads page. Instructions can be found on the BirdNET Runner page.
Instead of running BirdNET-Analyzer yourself, you just click on the BirdNET-Runner desktop icon and step through the prompts. Most times, you'll just be taking the defaults.
Once you have detections, you may want to verify a specific one by listening to the original clip. That's where BirdCall-Extractor comes in — a small Windows utility that extracts a short WAV clip from a larger recording, given the offset and duration from the BirdNET-Analyzer detection record. Just drop it on the desktop "green bird" icon, enter the offset, and the clip is saved alongside the original file ready for playback.
Original Focus: UK & Europe. Future Focus: Global
Although the tool was originally UK-centric, we have added support for US Letter page size reports and IUCN international red lists. More international features will be added in future based on user feedback.
More functionality will be added to both the website and toolchain as we learn more about what users need from us. We're learning too and are always interested in feedback (preferably constructive!).