Gary
Vollink.com

Apple Musicdb Format

Contents

Introduction

In macOS Catalina, Apple has replaced iTunes with individual applications called Books, TV, Podcasts and - the subject of this exploration - Apple Music.  Importantly, Apple Music no longer has an option to automatically keep an xml copy of the library available for third party tools (like Playlister).

This is a sub-project of Playlister, and is likely to become a stand-alone library that Playlister will be able to use if someone's music library is now managed by Apple Music under macOS Catalina.  As this is a hobby project, I do not expect to be finished quickly. 

Upon first run, Apple Music imports the original iTunes Library.itl file, and it also seemed to modify that file, though I don't have the ability to re-test to verify this.  After initial import, Apple Music does not open or do anything further with the iTunes Library.itl file.

Apple Music creates a folder called Music (inside $HOME/Music), and inside that creates a sparsebundle:
    /Users/gary/Music/Music/Music Library.musiclibrary

Inside Music Libary.musiclibrary are seven files.  Of these, three files have the extension, musicdb:
  Application.musicdb
  Library Preferences.musicdb
  Library.musicdb

musicdb File Format

I'm documenting the file format of the new musicdb format as I figure it out.  Unlike the iTunes itl format, all numbers (including IDs) are little endian, meaning they can be natively read by Intel/AMD x86 CPUs.  Large portions of this format are a mystery to me, and probably will continue to be.

I started this exploration with the Library.musicdb, because it has the most promising name, and is the largest of these files.  So far, all of the information I would expect to find is in this file, so I have put the most effort into making the documentation follow that file's format. 

The sections listed in the Table of Contents above are in the approximate order of the first appearance of each of those file sections (or problems to solve) in the Library.musicdb file.  Those sections that are listed but not documented can be, as safely as possible, skipped. 

While not complete, I believe that most of the information that could be pulled from the iTunes xml format can be recreated from the data-points documented below.

Envelope Header

* Dates are expressed in Seconds since 1-Jan-1904 Midnight.

Crypt Size

If Max Crypt Size is smaller than the File size, use it directly.

Crypt Size = File Size - Envelope Length - ((File Size - Envelope Length) mod 16)

Encrypted AND zlib Compressed Payload

Starting with the byte offset (Envelope length in Bytes), Crypt Size of the file needs to be decrypted using the standard AES128-ECB algorythm.  OpenSSL and macOS built-in libraries can do this in C/C++.  In Perl, use Crypt::Cypher.  A key is necessary to successfully unencrypt this. 

Once unencrypted, the payload must be expanded/inflated via zlib, including the rest of the file (if any) larger than the initial Max Crypt Size.

Back to Table of Contents.

Inner Library Sections

Almost all* sections follow this beginning pattern, with differences following these first 8 bytes.

These section headers follow the base pattern as described above:
hsma, hfma, plma, lama, iama, lAma, iAma, ltma, itma, lPma, and lpma.

* boma and ssma sections do not follow this pattern.

Back to Table of Contents.

hsma Section Boundary

Occurs multiple times (6).  It marks transitions between major sections of the file. 

Section SubTypes are
3 - First Boundary (first encrypted/compressed area), holds inner hfma.
6 - Holds Library Master (plma) and associated (boma) data.
4 - Holds lama and associated data.
5 - Holds lAma and associated data.
1 - Holds Track Master (ltma) and associated data.
3 - Holds Playlist Master (lPma) and associated data.

Associated Sections Length is counted from the beginning of this hsma section, and will point to either the next hsma section or the end of the file.

Back to Table of Contents.

hfma

Many parts of this are a repeat of the Outer Envelope hfma.

Back to Table of Contents.

plma Library Master

This appears to be the Library Master.  It occurs only once.  This section is followed by boma sections, the number of which is indicated at offset 8. 

Back to Table of Contents.

lama Master for iama

Appears only once.  Contains a count for iama sections in the library.  May be a list of Albums/Collections.

Back to Table of Contents.

iama

There are as many of these as specified in the lama record at offset 8.  May describe an Album/Collection.

Associated Sections length counts from the beginning of this iama, and points to the next iama or Boundary (hsma) section.

Back to Table of Contents.

lAma Master for iAma records

Appears only once.  Contains a count of total iAma sections that follow. May be a list of Artists/Bands.

Back to Table of Contents.

iAma Items

There are as many of these as specified in the lAma record at offset 8.  Seems to be just Artist Names/Artwork links, in the associated boma structures.

Back to Table of Contents.

ltma Tracks Items Master

Appears only once.  Contains a count of total tracks in library (or how many itma sections to look for).

Back to Table of Contents.

itma Track Item Header

Each itma section is the start of the description of a track. 

NOTES:
The 8 byte hex-presented IDs needs to be byte-swapped, like all other little-endian numbers.
An 8 byte ID that seems to be linked to iama Sections at offset 172
An 8 byte ID that seems to be linked to iAma Sections at offset 180
Track Persistent ID again:  The track persistent ID was only repeated on the first itma record, and was all zeroes in subsequent itma records.
There is a lot more information here that I haven't figured out. 

Back to Table of Contents.

lPma

Playlists Master or lPma shows up only once.  This lists the number of playlists (lpma) sections.

Back to Table of Contents.

lpma

Playlist Container or lpma exists for each Playlist.  This holds the number of associated (boma) sections for this playlist.

Observed as being followed by boma SubType 0x00C8 (Playlist Name), boma SubType 0x00CA (Unknown), optional boma SubType 0x00C9 (Smart Playlist Criteria), and a track count number of boma ipfa SubType 0x00CE (Playlist Track).

NOTE: There appears to be another date at offset (182) in some records, otherwise all zeroes, but I have not been able to figure out what it represents. 

Back to Table of Contents.

boma Data Container (common)

Every boma has a SubType, and what is inside each boma can be VERY different depending on that SubType.  The section length of a boma is indicated at a different offset from other section types.

Contrasting from the iTunes format:
It is useful to know that mhoh SubType numbers have the same purposes as the Apple Music boma SubTypes, though the offsets are slightly different. That said, iTunes has many more SubTypes, as that program could store much more than just Music files.

Back to Table of Contents.

boma SubType 0x0001 Track Numerics

• SubType:
  • 0x0001 - Bit Rate, Date Added, Date Modified, Song Time, File Size

*Date is translated from the numeric value based on Number of seconds since 1 January 1904, Midnight UTC. 

Back to Table of Contents.

boma Widechar SubTypes

• SubTypes:
  • 0x0002 - Track Title
  • 0x0003 - Album
  • 0x0004 - Artist
  • 0x0005 - Genre
  • 0x0006 - Kind
  • 0x0008 - Comment
  • 0x000C - Composer
  • 0x0020 - Sort Order Artist
  • 0x0021 - Sort Order Album Artist
  • 0x0022 - Sort Order Composer
  • 0x001B - Album Artist
  • 0x001E - Sort Order Track Name
  • 0x001F - Sort Order Album
  • 0x003F - Work Name (for Classical Tracks)
  • 0x0040 - Movement Name (for Classical Tracks)
  • 0x00C8 - Playlist Name
  • 0x012C - iama Album
  • 0x012D - iama Album Artist
  • 0x012E - iama Album Artist
  • 0x01F8 - Managed Media Folder
  • 0x02BE - Song Title (Application.musicdb)
  • 0x02BF - Song Artist (Application.musicdb)

Remember that the payload is in wide characters (UTF-16).  The resulting string length (in native) maybe as little as half.  In contrast with the iTunes itl format where there is an indicator of whether or not a payload is wide character or not. 

Back to Table of Contents.

boma Character SubTypes

• BOMA SubTypes:
  • 0x0038 - XML Block (unknown utility)
  • 0x00CD - XML Block (unknown utility)
  • 0x0192 - XML Block (unknown utility)
  • 0x02BC - XML Block (unknown utility)
  • 0x03CC - XML Block (unknown utility)

Technically this is UTF-8, but it can be safely copied bytewise.

Back to Table of Contents.

boma book SubTypes

• SubTypes:
  • 0x0042
  • 0x01FC
  • 0x01FD
  • 0x0200

boma book sections contain numbers and strings.  Strings are marked by 0x0101.  The four bytes prior to the 0x0101 will indicate the length of the string.  The byte following the 0x0101 will start the first character of that string.  These strings include the separate path components of the item.  The next string is 'file:///' The next string is 'Macintosh HD', or whatever the name of the install disk is.  The next string is the UUID of the install disk filesystem.  The final string is the path separator character, '/'.

After reading each string, it is important to re-align to a 4 byte offset.  For the string shown above, 5 bytes were read.  That means 3 bytes must be discarded before the next four bytes can be read.

boma book sections are very long, and most paths are repeated in another boma section (not separated). 

There is one string indicated by a 0x0901, with the same setup as the 0x0101 string, except it is further preceeded by a 0x01F5. 

There is one string indicated by a 0x0201, with the same setup as the 0x0101 string, except it is further preceeded by a 0x0501. 

Back to Table of Contents.

boma_ipfa

• SubType:
  • 0x00CE

This block holds a single Track of a playlist.  It also has an identifier that is sometimes also found in Application.musicdb, which I am referring to (for now) as the ipfa ID. 

Offset 64, ipfa ID 2, is usually zeroes, but when the ipfa is repeated here, it seems to indicate that this is the last Track ID within the playlist.

Back to Table of Contents.

boma Unknown

• SubTypes:
  • 0x00C9 - Found under a Playlist Container (lpma)
  • 0x00CA - Found under a Playlist Container (lpma)
  • 0x01F6 - Found under the Library Master (plma)

The preceeding are boma subtypes that follow the basic pattern at offsets (0), (8) and (12), but I have been unable to discern what data they hold.  Note, they also have differing lengths, so are probably not related.

Back to Table of Contents.

ssma

So far, I have only encountered this in Application.musicdb.  Marks (and pads) the end of the file. 

Back to Table of Contents.

Want To Help?

I am doing this work on a MacBook Air without enough space to put a large music library on here.  If you, the reader, are a user of macOS Catalina and happen to use Apple Music to store your music collection, I would love to run my assumptions above on your Library.musicdb file.

The file I am seeking is Music Library.musiclibrary which should automatically attach to an e-mail program as a .zip file. 

I would appreciate any musicdb files I can get.  Please send submissions to musicdb@vollink.com

Privacy:  The file(s) I'm asking for does not include any music.  It does include song file names, and how the user has rated and liked/disliked those songs, and all playlists saved.  The user's e-mail address may be embedded (I haven't found it yet, but it did exist in the older iTunes/itl format file).  Because the filenames include full paths, I will see the username of the Library (as in /Users/username/Music/iTunes/... ) I will NOT publish or share any files or information from any submission, unless I am explicitly given such permission.  I will use this to test the offsets and correllations I've found.  Unless explicitly asked to share, I will not retain any submitted file longer than 6 months (probably much less). 

This is a low traffic web site, and I will reply to any inquiry unless you specifically ask me not to reply.

Up to Table of Contents.

Acknowledgements

I would not have been able to get as far as I have without the fine work of Joseph Walton who produced titl.  That iTunes itl project in Java was key to my getting this file format cracked.

I also would not have been able to get as far as I have without the fine work of Jean Thomas who produced libitlp for reading iTunes itl format in C.  Despite all the section headers having different names, the formats (and tricks for hiding information) are remarkably similar.  Jean's faithful approach to parsing the actual format (and not an interpretation) helped immensely on what things to look for and what is probably safe to ignore.

Back to Table of Contents.