Mp3tag provides an internal Web Sources Framework which is parameterized through web sources description files.

Using these description files, you can import tag data from theoretically every web site which displays artist/album information via HTML (no JavaScript or PWAs) or provides an API via XML or JSON.

You can find many examples in the Web Sources Scripts category of the Mp3tag Community Forums. Q&A and helpful resources are available via the Web Sources Discussion category.

Specification of the Web Source file format

[Name]
Name of the web source, e.g., Discogs.com

[BasedOn]
Base URL of the web source, e.g., https://www.discogs.com

[IndexUrl]
Search URL where %s is replaced by the search criteria entered by the user, e.g., https://www.discogs.com/artist/%s

[AlbumUrl]
Result base URL (URL result from first search pass will be appended), e.g., https://www.discogs.com

[WordSeparator]
Character/string used instead of blanks within the search criteria entered by the user, e.g., %20 or +

[IndexFormat]
Format string for splitting the output buffer from the first search pass into different fields. %_url% is mandatory, e.g., %_url%|%album%|%type%|%label%
Use %_coverurl% with an URL to a reasonably sized image thumbnail to enable cover preview (should not be bigger than 250x250 pixels).

[SearchBy]
Field(s) which are offered as search criteria by the web source. You can use up to three different fields defined as triples of Field Name||%field%||&query=%s

Mp3tag displays one search field for each of the specified fields. Examples of this feature can be found with the standard Discogs web sources.

[Encoding]
Encoding used for all URLs. Can be utf-8, iso-8859-1, url, url-utf-8, or ansi (system codepage will be used).

[UserAgent]
If set to 1, Mp3tag uses its name and version number as user agent, e.g., Mp3tag/3.14.

[ParserScriptIndex]
This key contains a multi-line parser script (start with ...) which parses the search results page for different albums.

[ParserScriptAlbum]
This key contains a multi-line parser script (start with ...) which parses a web page found by the first search pass.

[Include]
This key references another web source description file which is included into this file. The keys from the referenced file overwrite the keys from the referencing file. Examples of this feature can be found with the standard Discogs.src which references Discogs.inc to reduce code duplication.

[MinAppVersionWin]
[MinAppVersionMac]
These keys reference the minimum app version numbers, e.g., 3.22 or 1.8.4, for Mp3tag for Windows and Mac that are required by the web source. As the format and language are ever-evolving, this helps to ensure that the running app supports everything that’s required by the web source to function properly.

[Settings]
This optional key references a .settings configuration schema that steers the configuration dialog for this web source. For details see Configuration Settings.

Parser Scripts

How to Start

First, you need to find a way to identify the lines which contain the interesting bits of information, for example the year of a release or the artist and album names. The Mp3tag parser sees output as lines and characters and you tell the parser how to move from the start to the places you want to extract as information.

Mp3tag uses a pointer which is positioned at the beginning of the file and which can be moved with several commands. There are different ways of moving the pointer to, e.g., the position where the year is referenced on a website

We can either move it down N lines or we can tell it to move down until it finds the text Year:. The first approach would be to use use the command MoveLine N (where N is the number of lines) or GotoLine N — this either means go down N lines from where you are or go to the Nth line from the top of the text.

To perform a search, the command would be FindLine "Year:", which finds the next line from where you are that contains the text Year:.

In all cases the pointer would be moved to the first character of the target line. From there we could tell the pointer to move N steps to the right with MoveChar N, or to move to the Nth character in the line using GotoChar N, or to position the pointer after the text Year: as in FindInLine "Year:".

Please note the difference between the FindLine and FindInLine commands: the former goes through lines from where you are to find a text and places the pointer at the beginning of the line, while the latter looks within the current line where the pointer is and positions it after the found text.

Now that we are where we want, we need tell Mp3tag to store the data. To do this, a Say command is used, but we have to find a way to tell it what to say and what not. In this case we want to output the rest of the line and so we use the SayRest command. An alternative would be the SayUntil " " command, which would output everything until a space character is found.

So, a script to find. extract, and output the year to a buffer name YEAR would look like this:

OutputTo "YEAR"
FindLine "Year:"
FindInLine "Year:"
SayNextNumber

You will notice that the example used the Find-Commands rather than the Move- or Goto-Commands. Whenever you have a chance to use a Find command, please do so, because websites tend to change over time. Using a script that relies on Find-Commands is more likely to survive a change in the raw data than one that relies on absolute positions.

For fields that have varying contents for tracks, e.g., TITLE, TRACK, or ISRC, Mp3tag expects the output buffer to contain the individual values separated and finalized by the pipe character |, e.g., Title 1|Title 2|Title 3|. Use $verticalBar() in case you need to emit the pipe character as content of a field.

JSON

The Web Source Framework supports parsing JSON-formatted data. So whenever the online tag source provides a JSON-based API, it’s recommended to use the API over the rendered website.

Debugging

Doing all this only theoretically can be a bit tricky and if you make an error counting lines or characters you might end up with quite unexpected results. To check what the parser is doing, you can add a Debug "on" "debug.out" command to the top of your script. This will give you an output file which will show you step by step what the parser is doing and why you end up with a given output.

You can use DebugWriteInput "debug-input.out" to see what data you are actually parsing and before you build your script step by step.

List of Parser Commands

You can reference contents of output buffers via %output% in all string parameters.

Configuration Settings

Starting with Mp3tag v3.22 and Mp3tag for Mac 1.8.4, a Tag Source can reference a configuration schema that is used to dynamically create a configuration dialog for the Tag Source.

The configuration schema needs to be stored in a .settings file, e.g., My Tag Source#Settings.settings and is referenced in the corresponding Tag Source via the [Settings] key, e.g., [Settings]=My Tag Source#Settings.settings.

This makes the configuration schema known to the Tag Source.

Configuration Schema Definition

The following example schema illustrates the overall structure and required keys for a configuration schema and lists all supported settings types and means for localization of configuration settings.

{
  "key": "test",
  "title": "Test Settings",
  "settings": [
	{
	  "type": "string",
	  "key": "testChoices",
	  "title": "Something to choose from",
	  "description": "Select the preferred item",
	  "choices": [
		"Item 1",
		"Item 2",
		"Item 3"
	  ],
	  "default": "Item 1"
	},
	{
	  "type": "string",
	  "key": "testEnter",
	  "title": "Something to enter text",
	  "description": "Enter some text",
	  "default": "Default Text"
	},
	{
	  "type": "number",
	  "key": "testNumber",
	  "title": "Something to enter a number",
	  "description": "Enter a number",
	  "default": 42
	},
	{
	  "type": "bool",
	  "key": "testCheck",
	  "title": "Something to check",
	  "description": "Check to enable",
	  "default": false
	},
	{
	  "type": "heading",
	  "key": "testHeading",
	  "title": "A bold-rendered heading text"
	},
	{
	  "type": "separator",
	  "key": "testSeparator",
	  "title": "An optional text that appears below the separator line"
	}
  ],
  "localizations": {
	"de": [
	  {
		"key": "test",
		"title": "Test Einstellungen"
	  },
	  {
		"key": "testChoices",
		"title": "Etwas zur Auswahl",
		"description": "Wählen Sie den gewünschten Eintrag"
	  },
	  {
		"key": "testEnter",
		"title": "Etwas zum Eingeben",
		"description": "Geben Sie den gewünschten Text ein"
	  },
	  {
		"key": "testNumber",
		"title": "Etwas zum Eingeben einer Zahl",
		"description": "Geben Sie die gewünschte Zahl ein"
	  },
	  {
		"key": "testCheck",
		"title": "Etwas zum Abhaken",
		"description": "Häkchen setzen zum Aktivieren"
	  },
	  {
		"key": "testHeading",
		"title": "Ein Überschriftentext in fetter Schrift"
	  },
	  {
		"key": "testSeparator",
		"title": "Ein optionaler Text, der unterhalb der Trennlinie erscheint"
	  }
	]
  }
}

Please note that localization items are optional and are referenced using the key from the individual items in the settings definition. The window title is referenced using the root key from the settings definition.

The language keys are using the ISO-639-1 language code.

Editing the Tag Source Configuration

To configure the settings, the user would use the dedicated menu item that is created at the end of the Tag Sources menu, separated from the actual Tag Sources. The example name My Tag Source#Settings.settings results in a menu item Settings in the My Tag Source submenu.

Choosing this menu item shows a dedicated dialog that is generated based on the individual settings schema.

The actual values that are configured by the user are stored in a separate file settings.json as simple key-value pairs, e.g.,

{
  "test": {
	"testChoices": "Item 1",
	"testCheck": false
  }
}

Note, that this settings file is used for the user settings of all Tag Sources. The individual objects use the key from the respective settings schema definition.

This ensures that you can add new configuration options with future versions of the Tag Source without the need to ask users to copy the additions to their settings file. It would just use the default if there is no value for the added key in the settings.json file.

Accessing Configuration Settings from a Tag Source

A dedicated function is used to access the individual settings in the Tag Source, where the key is used to access the setting which is compared to the given value.

IfVar "key" "value"
IfNotVar "key" "value"

With the examples above, you’d use

IfVar "testCheck" "true"
  ...
Else
 ...
Endif

or

IfNotVar "testChoices" "Item 1"
  ...
Else
 ...
Endif