17 posts tagged with "api"

Monthly Statistics​

During December the Metron Project added the following to its database:

• Users: 99
• Issues: 1,668
• Creators: 208
• Characters: 1,176

Thanks to everyone that contributed!

Website Development​

🎉 New Feature: User Collections​

This month I added a User Collections system that lets you track your personal comic book collection. You can manage your collection by adding or removing issues, mark things as read, and track comic condition using the CGC grading scale (0.5 to 10.0). The grading system supports professional grading companies like CGC, CBCS, and PGX, as well as user-assessed grades for raw comics.

There's also a star rating system (1-5 stars) for rating issues in your collection, format tracking for print and digital copies, and bulk operations to add entire series runs at once. If you're trying to complete runs, there's a missing issues tracker that shows you which issues you need and displays completion percentages for each series.

The feature includes collection statistics with visual charts, advanced filtering options to search by series, publisher, format, read status, grade, and more. There are also read-only API endpoints for accessing your collection data and statistics, along with documentation to help you get started.

📚 Reading Lists Enhancements​

Reading lists got some updates this month. You can now tag issues as Prologue, Core Issue, Tie-In, or Epilogue, with inline editing and visual badges to make it easier to see what's what. I also added a 5-star rating system for public reading lists, so you can see average ratings and filter lists by minimum rating.

There's also story arc integration that lets you add all issues from a story arc to a reading list in one go, which should save some time. I created a "reading list editor" group for managing Metron's curated lists - this allows trusted users to help curate without needing full staff privileges. If you're interested in being part of this group, just let me know.

The reading list API now supports browsing and retrieving lists with paginated results (50 items per page) and filtering options.

🔍 Search & Filtering Overhaul​

I spent some time improving the search functionality across the site. Series lists now support multi-word search with filters for type, publisher, imprint, year range, status, and volume. Issue lists have quick search with advanced filters for series info, dates, and IDs. Reading lists can be searched by name, creator, attribution source, and privacy status. User collections have quick search across series names and notes with multi-field filtering.

All the search interfaces now have collapsible advanced filter sections, preserve your filter state, show active filter indicators, and have better empty states. I tried to keep the UI patterns consistent across all of them.

🐛 Bug Fixes​

Week Views Date Calculation Fix (#437)​

Fixed a bug where WeekList, NextWeekList, and FutureList views were using class-level date calculations evaluated at import time. This meant the views would always show the same week (from when the server started) instead of updating dynamically. I moved the date calculations to per-request methods and switched to precise date range queries to fix this.

📱 Mobile Improvements​

Fixed a few mobile-specific issues this month: filter buttons now work properly on mobile devices, improved the home page responsiveness, fixed layout issues in home.html, and cleaned up button layouts across collection and reading list views.

🎨 UI/UX Enhancements​

Added a Reading List & Collections section to the home page, made the first information card full width, and improved the overall layout for better visual hierarchy. I also spent time optimizing database queries - reading list detail views went from around 20 queries down to 4-6 by using better prefetching and annotation strategies. Collection detail views now show cover images and descriptions with lazy loading.

🔒 Security & Validation​

Improved permission checks across reading list operations.

🔧 Technical Improvements​

Upgraded to Django 5.2.9 and added documentation for User Collections and Reading Lists, including user guides, technical documentation for developers, and API documentation. I also refactored and simplified some views with better inheritance patterns.

Looking Ahead​

Now that User Collections is up and running and Reading Lists have been improved, the plan is to refine these features based on user feedback and explore additional integration opportunities.

Desaad: A Comic Book Library Management System​

What is Desaad?​

Desaad is a self-hosted, web-based comic book library manager and reader built with Django that I've been working on over the past week or so. I created it to test and plan features for Metron that might be useful for other comic servers.

Key Features​

Library Management​

Desaad imports CBZ, CBR, and PDF comic archives and automatically pulls metadata from MetronInfo.xml files - things like series information, publishers, creators, characters, teams, universes, and story arcs. I'm not planning to add ComicInfo.xml support, since there are a lot of comic servers that already do.

It uses Darkseid for handling comic archives and extracting metadata.

Built-in Web Reader​

The web reader uses HTMX and automatically saves your reading position and progress, so you can pick up where you left off.

Reading Lists​

You can import reading lists from Metron using the API. The feature automatically tracks which issues are in your library and shows you which ones you're missing.

OPDS 1.2 Catalog Support​

You can access your comic library from mobile devices and e-readers using OPDS-compatible apps like Chunky Comic Reader (iOS), Panels (iOS), or Challenger Comics Viewer (Android).

I tried testing it with the Challenger app, but their OPDS support doesn't seem to work and is still marked as Beta. If anyone wants to test with Chunky or Panels, I'd appreciate hearing how it goes.

Auto-Import Functionality​

There's an import management command that you can run manually or set up to watch a directory for new comics.

I haven't optimized it much yet - the most expensive part is cover extraction, which I'll probably move to a background task at some point. I deployed it on a spare GMKtec NucBox G3 I had, and it took about 3 hours to import around 65,000 comics. Once I move cover extraction to a background task, I should be able to get that down to 1-2 hours.

Technical Foundation​

Desaad is built on Django 6.0 with SQLite for simplicity (though I'll probably switch to PostgreSQL at some point since SQLite has some performance limitations and missing features). The frontend uses the Bulma CSS framework with HTMX, and comic processing is handled by the Darkseid library. It's container-ready with Podman support and systemd integration.

It requires Python 3.13+ and uses the uv package manager for dependency management.

Important Considerations​

A few things to keep in mind about Desaad:

1. Single Metadata Source: It assumes all your comics use metadata from a single source (Metron, Comic Vine, etc.). Mixed metadata sources aren't supported.

2. MetronInfo.xml Format: Only supports the MetronInfo.xml metadata specification. Your comics need metadata written by software that implements this spec.

3. Under Active Development: The project is still under construction and not ready for production use. Features and APIs will change.

Who Should Use Desaad?​

Desaad might be a good fit if you're a developer or tester who prefers browser-based reading and wants to see possible new features like scrobbling. It works best if your collection uses metadata from a single source (Metron, Comic Vine, or Grand Comics Database). If your collection has mixed metadata from multiple sources, Desaad probably isn't the right tool for you.

Getting Started​

Setting up Desaad is pretty straightforward:

# Clone the repository
git clone https://codeberg.org/bpepple/desaad.git
cd desaad

# Configure environment
cp .env.example .env
# Edit .env with your settings

# Install dependencies and setup
uv sync
uv run python manage.py migrate
uv run python manage.py createsuperuser
uv run python manage.py collectstatic

# Run the development server
uv run python manage.py runserver

For production deployments, there are Podman compose configurations with systemd service integration.

Container Deployment​

If you're using Fedora Linux, the included Podman configuration should make deployment simple. It should work on other systems running Podman too, but I haven't tested that yet. To get started:

# Build and start
podman-compose up -d

# Create admin user
podman-compose exec web python manage.py createsuperuser

# Import comics
podman-compose exec web python manage.py import_comics

The application will be available at http://localhost:9000.

There's more information in the project's README.md, including how to create systemd service files.

OpenCollective​

A huge thank you to everyone who has contributed to our Open Collective! Your support makes a real difference in keeping the Metron Project running and growing.

What Your Contributions Support​

Funds from Open Collective go directly toward:

• Server hosting costs - Keeping the Metron website and API available
• Domain registration - Annual domain name renewals
• Future capacity increases - Scaling resources as the database and user base grows

All expenses are transparent and publicly viewable on our Open Collective page, so you can see exactly where every dollar goes.

Support the Project​

If you'd like to help keep the lights on and support continued development, contributions of any size are appreciated and help ensure Metron remains a free resource for the comic book community.

Anyway, that's everything for this month! Take care.

Monthly Statistics​

During November the Metron Project added the following to its database:

• Users: 30
• Issues: 1,042
• Creators: 158
• Characters: 531

Thanks to everyone that contributed!

Reading Lists​

Mea Culpa​

So... the last post I made introducing Reading List, unfortunately caused a bit of drama, since I forgot to attribute that the 13 initial reading lists were created from cbl files from the CBL-ReadingLists group. I'm sorry for this oversight and it wasn't my intention to give the impression they were created by me. I've since made a note on those reading lists attributing them to this group, and removed the import CLB functionality from the site.

Due to the aforementioned drama, I've had to call a bit of an audible in regard to the maintaining of the reading lists owned by the Metron user. I'm planning to work on adding support for non-admin users to edit these lists over the Holiday break and if you have an interest in helping to maintain them, please contact me.

One side-effect of my mistake is that it spurred on one of our community members to write a tool to retrieve the data directly from the reading lists source and check them against the site, and then produce a json file or if issues are missing generate a report that can be acted upon. Currently, I've got a management command to import these json files, but maybe I'll look at adding POST support to the API in the future.

Down the road, I'll look at adding arc-based bulk additions, other import options, and maybe an associated reading list fields.

Adding Series to Reading List​

I've added the option to add a series to a reading list, which should be easier than just adding individual issues.

API​

I've spent the last week or so working on the Reading List API, and it's finally been pushed to production. There is fairly detailed API documentation on the wiki, and there is also the Swagger API Documents.

I was originally planning to write a simple tool to generate reading lists from the API, but have held off due to:

• CBL files don't really have a schema.
• json-cbl-standard aren't currently designed to handle information from an API that well.

Mokkari​

A new version of Mokkari has been released that adds support for the new Reading List API, and also reworked the rate limiting code so that it produces more feedback to the user. If you have written an application that uses Mokkari, you'll want to modify your API calls to Metron to sleep when receiving RateLimitError. For example:

>>> import time
>>> from mokkari.exceptions import RateLimitError
>>> session = Session("username", "password")
>>> def fetch_with_rate_limit_handling(issue_id):
...     while True:
...         try:
...             return session.issue(issue_id)
...         except RateLimitError as e:
...             if "per minute" in str(e):
...                 # Minute limit - automatically wait and retry
...                 print(f"{e}")
...                 print(f"Waiting {format_time(e.retry_after)}...")
...                 time.sleep(e.retry_after)
...                 continue
...             elif "per day" in str(e):
...                 # Daily limit - ask user whether to wait or quit
...                 response = input(f"Wait {format_time(e.retry_after)}? (y/n): ")
...                 if response.lower() == 'y':
...                     time.sleep(e.retry_after)
...                     continue
...                 else:
...                     raise
...             else:
...                 raise
>>> issue = fetch_with_rate_limit_handling(1)

Metron-Tagger​

A couple of releases of Metron-Tagger were made since my last blog post. The big change was support for writing/reading metadata was added, but implemented differently than other tagging software which have overloaded the PDF metadata. I decided to treat PDFs, like we do with CBZ, by embedding the metadata.

After discussing this with AJ, he has also made a release of Comicbox supporting this method of writing comic metadata to a PDF.

The latest release has also improved the rate limiting, so that when the daily limit is exceeded, the user is asked whether they want to wait the reported number of minutes or quit processing the remaining files.

Cloudflare​

The site has been dealing with AiBots and scrapers quite a bit lately, and I'm giving serious thought to moving the site DNS over to Cloudflare to take advantage of the bot protection. I'm still working on what the migration plan would entail (name server changes, SSL certs, nginx, etc) and will make a decision once that's finished. If we do make the switch, the site will most likely be down for a while (assuming everything goes as planned), and I will put a notice on the site beforehand.

OpenCollective​

If you would like to help keep the lights on at the project, we have an account at Open Collective to defray the servers costs and help with increasing future server capacity.

Anyway, I think that's everything for this month! Take care.

Monthly Statistics​

During December the Metron Project added the following to its database:

• Users: 39
• Issues: 823
• Creators: 259
• Characters: 413

2024 Annual Statistics​

During 2024 the Metron Project added the following to its database:

• Users: 566
• Issues: 33,784
• Creators: 2,283
• Characters: 7,748

Thanks to everyone that contributed!

Grand Comics Database (GCD) ID Support​

Recently support for GCD ID's was added to the site, and we've currently linked about 50% of the issues on the site, and with luck, should be completed in the next week or so.

Potential January Projects​

I'm looking at some small projects to work on in January, and some potential ones I'm thinking about are:

• Add Legacy Numbering field
• Alternative Series Name field
• Advanced Resources (Series, Issues, etc) Search
• Resource ID URLS

Not sure which ones (or any) will be implemented, but I'm planning to dig into them a bit more in the next few weeks.

OpenCollective​

We've recently set up an account at Open Collective if you wish to help defray the server costs and help with increasing future server capacity.

Anyway, that's all I got for now, hope everyone had a great holiday and are enjoying New Years day. Take Care!

Monthly Statistics​

During November the Metron Project added the following to its database:

• Users: 31
• Issues: 2,210
• Creators: 104
• Characters: 426

Thanks to everyone that contributed!

Metron-Tagger v3.1.0​

I released a new version of Metron-Tagger today that adds the ability to migrate data from a comic that contains a ComicInfo.xml.

The conversion from ComicInfo.xml -> MetronInfo.xml does have some caveats though:

• Elements containing data with a comma in a delimited ComicInfo element. If for example, the Title element contained a story call "Babs: True Blue, I Love You" and it was originally written with a tool other than Metron-Tagger it will split the story into two element items: Babs: True Blue and I Love You .
• Ratings. We map these to the enums used in MetronInfo, but it's not perfect since the ComicInfo documentation isn't terribly clear how it defines it values, so if you feel our mapping is incorrect please notify us and explain what it should be. Also, I've seen some users use non-valid values which obviously won't work.
• Series Format. We've also done our best to map the series format, but this is even less defined the age rating since this is just a text element in the schema, and could be anything. We've used the values in Comic-Tagger to handle the conversion, but this isn't perfect.

To migrate your ComicInfo.xml to MetronInfo.xml without any online lookup would simply run:

metron-tagger --migrate /path/to/comics

If you wanted to tag you comics with data from the Metron Comic Book Database, and migrate the data from the existing ComicInfo.xml for those comics not on Metron, you would run:

metron-tagger -om --migrate /path/to/comics

If you run across any bugs or have suggestions for improvements, please file a Bug Report or Feature Request.

Server Upgrade​

Finally got around to upgrading the server, and it was fairly uneventful. Yay! 🎉

OpenCollective​

The main repository has finally reached 100 stars, and so most likely in the next week or two, I'll set up a funding option at Open Collective to help defray the server costs and help with increasing future server capacity.

Future​

Now the bulk of the work is done with getting the MetronInfo out the door and some initial tooling to generate them, I'm planning to work on some low-hanging bugs with the website over the next month or so, before starting any large scale projects.

Anyway, that's all I've got for this month. Take care!

New Indexed Issues Milestone​

The Metron Project reached a milestone today by having 65,000 issues added to its database. I wish to give big thanks to everyone who has helped with the project. Thanks!

POST API​

Been spending the last few months working on a POST API to make it easier to add/update information on Metron. Currently it's only available to users with Adminstrators permissions, but later this year I'll look at the feasability of making it more readily available.

Comic Vine ID's​

Another project I've been working on is adding Comic Vine identification numbers to the various resource items. This should help users match items between the two data sources. Currently, about 40% of the issues have been updated with this information and with any luck the majority of issues will be finished by the end of the month.

ComicTagger​

One of the developers of Comic Tagger has been working on adding support for tagging digital comics with data from Metron, and hopefully in the next couple of months that merged.

Did a quick release this morning of the python wrapper for the Marvel API, Esak, which adds an option to have the cache results expire after a user-defined number of days. To update to the latest version just run the following command:

pip3 install -U --user esak

The Metron Project reached a milestone today by having 50,000 issues added to its database. I wish to give big thanks to everyone who has helped with the project. Thanks!