9 posts tagged with "comicvine"

A quick follow-up to the v1.0 announcement from two years ago: MetronInfo v1.1 is out, adding two new optional elements that came out of feedback since the initial release.

Monthly Statistics​

During February the Metron Project added the following to its database:

• Users: 51
• Issues: 2,674
• Creators: 526
• Characters: 879

Thanks to everyone that contributed!

Recent Work​

One of my goals for 2025 was to make the project less dependent on me, and for most of this month I've been working on the tooling to allowing some users to use the POST API to add comics to the database. This will allow other folks to handle requests to add series to Metron. With luck, I should finish next month, and I'll start looking at what users to promote.

Comic Book Reading Lists​

Recently, I've been looking at Comic Book Reading Lists since a bug report was opened to add support for generating them. Currently, users use CBL's that are imported into their comic server of choice (Kavita, Codex, etc.), which seems non-optimal primarily since, in essence, it's another database of information. Now, this makes sense, since for a long time the only source of usable data via an API was Comic Vine, which hasn't had a developer presence in years, and having a new feature added to their site is extremely unlikely.

I think a better solution would be for the other databases that offer an API (GCD & Metron) to have the option for users to create reading lists (similar to MovieDB) and have that data available either by generating a file or through an API call. Now, to do this we would need to work with the comic server developers to determine exactly what data they would need, though it would probably look similar to this:

{
  "arc": {
    "name": "Foo Bar Wars"
  }
  "issues": {
    "issue": {
      "id": {
      "cv_id": 9999
      "metron": 5678
      "gcd": 9856
      }
      "index": 0
    }
    "issue": {
      "id": {
      "cv_id": 12345
      "metron": 9999
      "gcd": 123
      }
      "index": 1
    }
  }
}

The current standard (CBL) and the recent json-cbl-standard contains a look of information that I can't see a comic server needing, primarily all the series-level info which the comic servers should already have in their databases. Of course, I'd love to hear from any developer in this space to verify that.

Once the basic functionality was added, importing any existing CBL into GCD or Metron should be fairly trivial, since Metron links other sites issue id's (GCD would be able to use Metron's API to link since they don't track this information).

Anyway, those are some of my quick thoughts on what the future of Comic Book Reading Lists could look like, and if anyone has interest in working on this potential future project, contact me on GitHub or Matrix.

Rate Limiting​

One thing I've been debating is whether to modify our Rate Limiting policy. The primary reason for this is there are a lot of user that hammer the API for 24+ hours straight for days on end, and I'm contemplating adding a total number of API hits per day (maybe something like 10,000) in addition to the 30/hour to ease the server load.

Now doing this will make it a bit harder implement to write software for the API since the developer will need to track the API rate between sessions, but there are libraries like pyrate-limiter which use a SQLite or Redis backend to make this easier to implement.

Anyway, this is something I'm just mulling over and if you want to weigh-in you can on Matrix or in a GitHub Discussion.

OpenCollective​

We've recently set up an account at Open Collective if you wish to help defray the server costs and help with increasing future server capacity. Any help would be greatly appreciated.

Anyway, that's all I got for this month. Take care!

Monthly Statistics​

During January the Metron Project added the following to its database:

• Users: 52
• Issues: 1,038
• Creators: 326
• Characters: 326

Thanks to everyone that contributed!

Grand Comics Database (GCD) IDs​

Recently support for GCD ID support was added and currently about 92% of the issues on the site has had them added.

Alternative / Legacy Numbering​

Recently, Metron added a new field to track alternative / legacy numbering, which are used primarily with DC & Marvel comics with their long-running characters.

In addition, I've also made changes to the filter so that issue numbers will be not case-sensitive when searched, which will improve matching for issue like The Amazing Spider-Man (2022) #65.DEATHS.

I'm planning to add a new param to the API that will allow you to search the number and alternative number fields at the same time, but there isn't an ETA when that will be added.

URL Redirects​

Recently, support for ID URL Redirects was added, so if you know the Metron ID number for an issue it will redirect to the correct page. For example, https://metron.cloud/issue/133777/ will redirect you to the detail page for The Amazing Spider-Man (2022) #66.

Digital Releases​

Recently a decision was made not to include digital releases that are 5 pages or less, like the Marvel Legacy Primer Pages. If you have questions whether a release should be added or not, you can ask on the Discussions Page or on Matrix.

International Support​

In the effort to add support to the site for Non-US publishers, a country field has been added to the Publisher model, though only the United States will be valid. Modifying the price field for an issue to handle different currencies still needs to be done, but with some luck I'll find some time to do it this year.

OpenCollective​

We've recently set up an account at Open Collective if you wish to help defray the server costs and help with increasing future server capacity.

Anyway, I think that covers most of the changes to the site that were made this month. Take care!

Monthly Statistics​

During December the Metron Project added the following to its database:

• Users: 39
• Issues: 823
• Creators: 259
• Characters: 413

2024 Annual Statistics​

During 2024 the Metron Project added the following to its database:

• Users: 566
• Issues: 33,784
• Creators: 2,283
• Characters: 7,748

Thanks to everyone that contributed!

Grand Comics Database (GCD) ID Support​

Recently support for GCD ID's was added to the site, and we've currently linked about 50% of the issues on the site, and with luck, should be completed in the next week or so.

Potential January Projects​

I'm looking at some small projects to work on in January, and some potential ones I'm thinking about are:

• Add Legacy Numbering field
• Alternative Series Name field
• Advanced Resources (Series, Issues, etc) Search
• Resource ID URLS

Not sure which ones (or any) will be implemented, but I'm planning to dig into them a bit more in the next few weeks.

OpenCollective​

We've recently set up an account at Open Collective if you wish to help defray the server costs and help with increasing future server capacity.

Anyway, that's all I got for now, hope everyone had a great holiday and are enjoying New Years day. Take Care!

Monthly Statistics​

During November the Metron Project added the following to its database:

• Users: 31
• Issues: 2,210
• Creators: 104
• Characters: 426

Thanks to everyone that contributed!

Metron-Tagger v3.1.0​

I released a new version of Metron-Tagger today that adds the ability to migrate data from a comic that contains a ComicInfo.xml.

The conversion from ComicInfo.xml -> MetronInfo.xml does have some caveats though:

• Elements containing data with a comma in a delimited ComicInfo element. If for example, the Title element contained a story call "Babs: True Blue, I Love You" and it was originally written with a tool other than Metron-Tagger it will split the story into two element items: Babs: True Blue and I Love You .
• Ratings. We map these to the enums used in MetronInfo, but it's not perfect since the ComicInfo documentation isn't terribly clear how it defines it values, so if you feel our mapping is incorrect please notify us and explain what it should be. Also, I've seen some users use non-valid values which obviously won't work.
• Series Format. We've also done our best to map the series format, but this is even less defined the age rating since this is just a text element in the schema, and could be anything. We've used the values in Comic-Tagger to handle the conversion, but this isn't perfect.

To migrate your ComicInfo.xml to MetronInfo.xml without any online lookup would simply run:

metron-tagger --migrate /path/to/comics

If you wanted to tag you comics with data from the Metron Comic Book Database, and migrate the data from the existing ComicInfo.xml for those comics not on Metron, you would run:

metron-tagger -om --migrate /path/to/comics

If you run across any bugs or have suggestions for improvements, please file a Bug Report or Feature Request.

Server Upgrade​

Finally got around to upgrading the server, and it was fairly uneventful. Yay! 🎉

OpenCollective​

The main repository has finally reached 100 stars, and so most likely in the next week or two, I'll set up a funding option at Open Collective to help defray the server costs and help with increasing future server capacity.

Future​

Now the bulk of the work is done with getting the MetronInfo out the door and some initial tooling to generate them, I'm planning to work on some low-hanging bugs with the website over the next month or so, before starting any large scale projects.

Anyway, that's all I've got for this month. Take care!

I'm happy to announce the v1.0 of the MetronInfo schema has been finalized and is now ready to be used by developers and consumers. So, I'll start off by answering some questions (most of this will be from the projects README).

What is it?​

MetronInfo is a new xml document type that fixes some deficiencies that exist in the current de facto xml file, ComicInfo.xml, to store metadata for a comic. Some of our improvements include:

• Addition elements to track information that are not included with the current standard.
• Better handling of data types. For instance, instead of using delimited strings we use arrays of elements.
• Elements to track where the data was obtained from.
• id attributes for resources (Characters, Creators, etc.), so items with different names can be associated correctly.

Is the schema only for the Metron Database?​

No, the schema only has Metron in the name since almost every other format has Comic in the name, and the originating author hates naming projects, so he went with the simplest choice. 😄 It was designed to be used for any of the comic resources (Comic Vine, AniList, etc.)

Where can I find the schemas?​

Version 1.0 of the schema is located in schema directory

Is there documentation for it?​

Yes, there is documentation describing the elements usage and also a Matrix to help with age rating mapping.

How can I validate my XML?​

It's recommended that any software that writes the XML make use of the schema to validate, so consumers of the XML document can be sure of its data. The schema use XSD 1.1, so you need to make sure your validation code uses that instead of XSD 1.0.

For example to validate the XML in python:

from pathlib import Path
from xmlschema import XMLSchema11, XMLSchemaValidationError

xsd = Path("/home/user/MetronInfo.xsd")
xml = Path("/home/user/MetronInfo.xml")

schema = XMLSchema11(xsd)
try:
    schema.validate(xml)
except XMLSchemaValidationError as e:
    print(f"Failed to validate XML: {e!r}")
    exit(1)

# Code to write / read the xml file

What software currently supports it?​

Currently, the following software does:

• Metron-Tagger - Commandline tool to tag comic with metadata ( MetronInfo or ComicInfo) from Metron Comic Book Database.

If you are a developer that has added support for MetronInfo.xml to your software, please create a PR to update the README or contact me.

Future​

One of the goals of the project was to be responsive to the needs of the community, so if you have suggestion for improvements to the schema please file a feature request, bug report, or start a discussion.

Thanks​

I'd like to thank everyone that has contributed to the schema, and in particular AJ, Buried-In-Code, and Joe who have helped out a lot in the last few months.

Ok, been fairly busy the last couple of months so naturally blogging has taken a back-burner. So, here's a summary of what's been going on the last few months.

Monthly Statistics​

During August the Metron Project added the following to its database:

• Users: 41
• Issues: 1,377
• Creators: 147
• Characters: 346

During September the following was added:

• Users: 100
• Issues: 1,330
• Creators: 124
• Characters: 379

During October the following was added:

• Users: 51
• Issues: 1,996
• Creators: 122
• Characters: 740

Comic Vine Outage​

Comic Vine went down at the end of September for about 6 days, so we saw an influx of new users which necessitated in increasing our server's capacity. One thing this episode brought to light out was the need for the project to accept donations for future infrastructure costs (since I'm currently covering the server costs).

To help us set this up it would be greatly appreciated if you could star the repository since one of the requirements to set up an account on Open Collective is to have 100 stars.

Server OS Upgrade​

I had planned to upgrade the server in August, but y'know... life. Anyway, I'm tentatively planning to upgrade Metron's operating system on November 16th, so there might be some downtime (most likely a few hours) when I do that. I'll post a notification on the site for the week leading to the upgrade.

MetronInfo.xml​

The last couple of months there has been a bunch of work done on finalizing the MetronInfo.xsd by Joe, AJ, Buried-In-Code and myself, and I believe there are just a couple of outstanding items that need a resolution. Once that is done, I'll write up a announcement blog post.

Anyway, that's all I've got for this month. Take care!

New Indexed Issues Milestone​

The Metron Project reached a milestone today by having 65,000 issues added to its database. I wish to give big thanks to everyone who has helped with the project. Thanks!

POST API​

Been spending the last few months working on a POST API to make it easier to add/update information on Metron. Currently it's only available to users with Adminstrators permissions, but later this year I'll look at the feasability of making it more readily available.

Comic Vine ID's​

Another project I've been working on is adding Comic Vine identification numbers to the various resource items. This should help users match items between the two data sources. Currently, about 40% of the issues have been updated with this information and with any luck the majority of issues will be finished by the end of the month.

ComicTagger​

One of the developers of Comic Tagger has been working on adding support for tagging digital comics with data from Metron, and hopefully in the next couple of months that merged.

So, one of the new projects I've been working on in my spare time is a new comic book xml schema called MetronInfo.

Currently, most applications use ComicInfo which originated from the ComicRack application, but due to it's history this format has some limitations like:

• Poor handling of data. For example, cover date information that is split into separate integer types, instead of just using a date type.
• Limited types metadata stored. For example, when tagging a comic book archive with an application like ComicTagger it would be useful to know where the metadata information was retrieved from (i.e. Comic Vine, GCD, Metron, etc.) and the resources identification number.
• Lack of documentation. The Anansi Project is working on improving this, tho.

For now, the Anansi Project is working on documenting and extending (non-breaking changes) the ComicInfo schema, but in my opinion that time spent would be better spent on creating a new format not tied to a dead application. Due to the fairly unstructured nature of comic book archives it's trivial to tag one with the existing ComicInfo format and also a new format. That way users who use applications that only support ComicInfo.xml can still use them, but also gives them the option to use applications that could support the MetronInfo format.

My goals for MetronInfo.xml are:

• Fix some of the deficiencies of the ComicInfo format.
• Add information to make it easier for Plex-like application to identify data (series, character, etc/) when importing comics into their application.
• Document the elements clearly so their use is not ambiguous.

Currently I'm working right now on finalizing the new format, and would appreciate any help or feedback from the comic book community in this. Feel free to leave suggestions either at the Discussions Page or contact use over on Matrix.

Once the new format is finalized I plan on writing a tool that will create a MetronInfo.xml file for any existing comic book archive that was tagged with ComicTagger or Metron-Tagger, and also retrieve any missing data from the source of information (Metron or Comic Vine).