[M] Gary
Vollink.com

Apple MusicDB Format

Contents

Introduction

In macOS Catalina, Apple has replaced iTunes with individual applications called Books, TV, Podcasts and - the subject of this exploration - Apple Music.  Importantly, Apple Music no longer has an option to automatically keep an xml copy of the library available for third party tools (like Playlister).

This is a sub-project of Playlister, and at some point, will likely to become a stand-alone library that Playlister will be able to use if someone's music library is now managed by Apple Music under macOS Catalina or higher. 

This is a hobby project, and I have pretty much stopped working on it, mostly because iTunes on Windows still exists, and still exports the xml file, automatically on any library change.  As implied, I stopped managing my playlists on my macOS hardware.

musicdb File Layout

Upon first run, Apple Music imports an original iTunes Library.itl file, if one exists.  After initial import, Apple Music does not open or do anything further with the iTunes Library.itl file.

Apple Music creates a folder called Music (inside $HOME/Music), and inside that creates a sparsebundle:
    /Users/gary/Music/Music/Music Library.musiclibrary

Inside Music Libary.musiclibrary are seven files.  Of these, three files have the extension, musicdb:
  Application.musicdb
  Library Preferences.musicdb
  Library.musicdb

musicdb File Format

Some portions of this format remain a mystery.  Because of this, this information will NOT be enough to create or even manipulate a muscidb externally. 

I started this exploration with the Library.musicdb, because it has the most promising name, and is the largest of these files.  So far, all of the information I would expect to find is in this file, so I put the most effort into making the documentation follow that file's format. 

The sections listed in the Table of Contents above are in the approximate order of the first appearance of each of those file sections (or problems to solve) in the Library.musicdb file.  Those sections that are listed but not documented can be, as safely as possible, skipped. 

While not complete, I believe that most of the information that could be pulled from the iTunes xml format can be recreated from the data-points documented below.

Items from the XML I have not been able to find...
Track -- Track ID (Track primary key in XML)
Track -- Sample Rate
Track -- Disc Number
Track -- Disc Count
Track -- Track Count
Track -- Track Type
Track -- File Folder Count ?
Track -- Library Folder Count ?

Endian Handling

Everything documented here assumes Little Endian binary formatting, as I have not yet encounterd a musicdb library that is stored in Big Endian byte order.  However, iTunes would save with either format, depending on the section or sometimes the hardware (as Apples used to run RISC / POWER architecture).  I added this section because I do NOT know whether the Endian of this file might flip under the new Apple silicon products.  With that in mind, I recommend being ready for such flips.

Each section is documented here with its Little Endian format.  I suspect that, like iTunes, the four header identification bytes will show up in reverse order on a section that has been saved with Big Endian formatting. 

My approach is to keep an Endian tracking value along with each section I read, and to stay careful to handle byte-swaps that are needed (or pass the appropriate flag to the unpack() calls).  For Widechar sections, I have an Endian check while decoding the UTF-16 bytestream.  For Little Endian, decode UTF-16le, but for Big Endian, decode UTF-16be.

If you run Apple Music on M1 hardware, I want a copy of your library, take a look below.

Envelope Header

* Dates are expressed in Seconds since 1-Jan-1904 Midnight.

Crypt Size

If Max Crypt Size is smaller than the File size, use it directly.

Crypt Size = File Size - Envelope Length - ((File Size - Envelope Length) mod 16)

Encrypted AND zlib Compressed Payload

Starting with the byte offset (Envelope length in Bytes), Crypt Size of the file needs to be decrypted using the standard AES128-ECB algorythm.  OpenSSL and macOS built-in libraries can do this in C/C++.  In Perl, use Crypt::Cypher.  A key is necessary to successfully unencrypt this. 

Once unencrypted, the payload must be expanded/inflated via zlib, including the rest of the file (if any) larger than the initial Max Crypt Size.

Back to Table of Contents.

Inner Library Sections

Almost all* sections follow this beginning pattern, with differences following these first 8 bytes.

These section headers follow the base pattern as described above:
hsma, hfma, plma, lama, iama, lAma, iAma, ltma, itma, lPma, and lpma.

* boma and ssma sections do not follow this pattern.

Back to Table of Contents.

hsma Section Boundary

Occurs multiple times (6).  It marks transitions between major sections of the file. 

Section SubTypes are
3 - First Boundary (first encrypted/compressed area), holds inner hfma.
6 - Holds Library Master (plma) and associated (boma) data.
4 - Holds lama and associated data.
5 - Holds lAma and associated data.
1 - Holds Track Master (ltma) and associated data.
3 - Holds Playlist Master (lPma) and associated data.

Associated Sections Length is counted from the beginning of this hsma section, and will point to either the next hsma section or the end of the file.

Back to Table of Contents.

hfma

Any parts of this with data are a duplicate of the Outer Envelope hfma.

Back to Table of Contents.

plma Library Master

This appears to be the Library Master.  It occurs only once.  This section is followed by boma sections, the number of which is indicated at offset 8. 

Back to Table of Contents.

lama Master for iama

Appears only once.  Contains a count for iama sections in the library.  May be a list of Albums/Collections.

Back to Table of Contents.

iama

There are as many of these as specified in the lama record at offset 8.  May describe an Album/Collection.

Associated Sections length counts from the beginning of this iama, and points to the next iama or Boundary (hsma) section.

Back to Table of Contents.

lAma Master for iAma records

Appears only once.  Contains a count of total iAma sections that follow. May be a list of Artists/Bands.

Back to Table of Contents.

iAma Items

There are as many of these as specified in the lAma record at offset 8.  Seems to be just Artist Names/Artwork links, in the associated boma structures.

Back to Table of Contents.

ltma Tracks Items Master

Appears only once.  Contains a count of total tracks in library (or how many itma sections to look for).

Back to Table of Contents.

itma Track Item Header

Each itma section is the start of the description of a track. 

NOTES:
The 8 byte hex-presented IDs needs to be byte-swapped, like all other little-endian numbers.
An 8 byte ID that seems to be linked to iama Sections at offset 172
An 8 byte ID that seems to be linked to iAma Sections at offset 180
Track Persistent ID again:  The track persistent ID was only repeated on the first itma record, and was all zeroes in subsequent itma records.
There is a lot more information here that I haven't figured out. 

Back to Table of Contents.

lPma

Playlists Master or lPma shows up only once.  This lists the number of playlists (lpma) sections.

Back to Table of Contents.

lpma

Playlist Container or lpma exists for each Playlist.  This holds the number of associated (boma) sections for this playlist.

Observed as being followed by boma SubType 0x00C8 (Playlist Name), boma SubType 0x00CA (Unknown), optional boma SubType 0x00C9 (Smart Playlist Criteria), and a track count number of boma ipfa SubType 0x00CE (Playlist Track).

NOTE: There appears to be another date at offset (182) in some records, otherwise all zeroes, but I have not been able to figure out what it represents. 

Back to Table of Contents.

boma Data Container (common)

Every boma has a SubType, and what is inside each boma can be VERY different depending on that SubType.  The section length of a boma is indicated at a different offset from other section types.

Contrasting from the iTunes format:
It is useful to know that mhoh SubType numbers have the same purposes as the Apple Music boma SubTypes, though the offsets are slightly different. That said, iTunes has many more SubTypes, as that program could store much more than just Music files.

Back to Table of Contents.

boma SubType 0x0001 Track Numerics

• SubType:
  • 0x0001 - Bit Rate, Date Added, Date Modified, Song Time, File Size

*Date is translated from the numeric value based on Number of seconds since 1 January 1904, Midnight UTC. 

Back to Table of Contents.

boma Widechar SubTypes

These sections are encoded UTF-16, which is Endian sensitive. 

• SubTypes:
  • 0x0002 - Track Title
  • 0x0003 - Album
  • 0x0004 - Artist
  • 0x0005 - Genre
  • 0x0006 - Kind
  • 0x0007 - (not sure)
  • 0x0008 - Comment
  • 0x000C - Composer
  • 0x000E - Grouping (Classical)
  • 0x0012 - Episode Comment (maybe)
  • 0x0016 - Episode Synopsis
  • 0x0018 - Series Title
  • 0x0019 - Episode Number
  • 0x001B - Album Artist
  • 0x001C - Series (Unknown Info)
  • 0x001E - Sort Order Track Name
  • 0x001F - Sort Order Album
  • 0x0020 - Sort Order Artist
  • 0x0021 - Sort Order Album Artist
  • 0x0022 - Sort Order Composer
  • 0x002B - Licensor/Copyright Holder ?
  • 0x002E - (not sure)
  • 0x0033 - Series Synopsis
  • 0x0034 - Flavor String
  • 0x003B - Email (purchaser?)
  • 0x003C - Name (purchaser?)
  • 0x003F - Work Name (for Classical Tracks)
  • 0x0040 - Movement Name (for Classical Tracks)
  • 0x00C8 - Playlist Name
  • 0x012C - iama Album
  • 0x012D - iama Album Artist
  • 0x012E - iama Album Artist
  • 0x012F - Series Title
  • 0x01F4 - Unknown 64x4b Hex String
  • 0x01F8 - Managed Media Folder
  • 0x01FE - Unknown 64x4b Hex String
  • 0x02BE - Song Title (Application.musicdb)
  • 0x02BF - Song Artist (Application.musicdb)

Back to Table of Contents.

boma UTF-8 Short Header SubTypes

• BOMA SubTypes:
  • 0x0036 - XML Block (unknown utility)
  • 0x0038 - XML Block (unknown utility)
  • 0x0192 - XML Block Artwork URL

Back to Table of Contents.

boma UTF-8 Long Header SubTypes

• BOMA SubTypes:
  • 0x001D - XML Block (unknown utility)
  • 0x00CD - XML Block (unknown utility)
  • 0x02BC - XML Block (unknown utility)
  • 0x03CC - XML Block (unknown utility)

Back to Table of Contents.

boma book SubTypes

• SubTypes:
  • 0x0042
  • 0x01FC
  • 0x01FD
  • 0x0200

boma book sections contain numbers and strings.  Strings are marked by 0x0101.  The four bytes prior to the 0x0101 will indicate the length of the string.  The byte following the 0x0101 will start the first character of that string.  These strings include the separate path components of the item.  The next string is 'file:///' The next string is 'Macintosh HD', or whatever the name of the install disk is.  The next string is the UUID of the install disk filesystem.  The final string is the path separator character, '/'.

After reading each string, it is important to re-align to a 4 byte offset.  For the string shown above, 5 bytes were read.  That means 3 bytes must be discarded before the next four bytes can be read.

boma book sections are very long, and most paths are repeated in another boma section (not separated). 

There is one string indicated by a 0x0901, with the same setup as the 0x0101 string, except it is further preceeded by a 0x01F5. 

There is one string indicated by a 0x0201, with the same setup as the 0x0101 string, except it is further preceeded by a 0x0501. 

Back to Table of Contents.

boma_ipfa

• SubType:
  • 0x00CE

This block holds a single Track of a playlist.  It also has an identifier that is sometimes also found in Application.musicdb, which I am referring to (for now) as the ipfa ID. 

Offset 64, ipfa ID 2, is usually zeroes, but when the ipfa is repeated here, it seems to indicate that this is the last Track ID within the playlist.

Back to Table of Contents.

boma Video Info

• SubType:
  • 0x0024

Offset 68 may indicate the frame-rate?

boma Unknown

• SubTypes:
  • 0x00C9 - Found under a Playlist Container (lpma)
  • 0x00CA - Found under a Playlist Container (lpma)
  • 0x01F6 - Found under the Library Master (plma)

The preceeding are boma subtypes that follow the basic pattern at offsets (0), (8) and (12), but I have been unable to discern what data they hold.  Note, they also have differing lengths, so are probably not related.

Back to Table of Contents.

ssma

So far, I have only encountered this in Application.musicdb.  Marks (and pads) the end of the file. 

Back to Table of Contents.

Want To Help?

Since putting up this page, I have received a handful of submissions from folks who want to see this work further improved.  As of late 2021, I stopped finding any new insights.  As of 2022, I have put this project on a deep pause.

If you think you might be able to help, I make my personal diagnostic script available to you (though I do not publish the decryption key).  More information on my personal GitLab:

MusicDB-POC

If you run the above, and figure out some data-point that I've skipped in my layout above, send it to me.  I'll be happy to publish it here, and credit you (as I have others).

Please send submissions to musicdb@vollink.com

This is a low traffic web site, and I will reply to any inquiry unless you specifically ask me not to reply.

Up to Table of Contents.

I Cannot Save Your Corrupted Library

After many e-mails asking if I can save a corrupted Apple Music database file, the answer is no.  Apple decided to both compress the entire file PLUS encrypt the first large chunk of it.  If the start of the file is corrupted, then the encryption will be broken, and the uncompression will never be able to find its place.  If the file is corrupted later, the compression will lose it's place, leaving the remainder of the file unreadable.

I have published the program that I use to poke through my own musicdb files as well as submissions that I get, so feel free to go for it.

MusicdB-POC

Back to Table of Contents.

Acknowledgements

I would not have been able to get as far as I have without the fine work of Joseph Walton who produced titl.  That iTunes itl project in Java was key to my getting this file format cracked.

I also would not have been able to get as far as I have without the fine work of Jean Thomas who produced libitlp for reading iTunes itl format in C.  Despite all the section headers having different names, the formats (and tricks for hiding information) are remarkably similar.  Jean's faithful approach to parsing the actual format (and not an interpretation) helped immensely on what things to look for and what is probably safe to ignore.

In early 2021, a developer named rinsuki reached out to me with a number of additional data-points that are present in the header/hfma, and additionally clarified the meaning of a duplicate ID I had spotted in some musicdb libraries.  This includes header offsets 12, 14, 56, 68, 72, 76, 80, 88, 92, 104 and the sometimes duplicated Library ID at 108.

Late July 2021, another developer, François Lamboley, shared a library with a number of unique features. This library had six new BOMA UTF-16 subtypes, but most importantly, included Emoji within some names, which led to a revelation that UTF-16 is the strict encoding of the Widechar BOMA sections.

August 2021, I finally got a submission from an M1 Apple, from twitter: @GeekAndDad.  This library, like the ones from x86_64, is entirely Big Endian formatting.  I would like to see another library from someone else before actually saying that it won't be necessary to have Endian flip detection in an implimentation for a reader, but this is looking positive.

Last, but not least, Paul and Isaac, who each read this page and were the first two to send me Libraries to explore.  This led to a major expansion of boma subtypes documented.  More important, it took that second person reaching out and my stub programs failing to read the library to make me re-visit a header offset location (Max Crypt Size), and realize it was wrong the whole time (now corrected on this page). 

Back to Table of Contents.