Greg Wilder

Composer | Pianist | Music Informatics

How Does Isomer Work?

How Does Isomer Work?

posted on

Isomer is a suite of software tools that produce abstracted representations of the characteristic trends expressed in existing musical models. The system then uses these observed trends to create query and transformation algorithms, with the ultimate goal of generating of new/hybrid materials.

Isomer’s ten software modules are separated into two categories: representation and processing. Summary descriptions of each module are presented below.

Model Representation

Audio
Isomer-Audio analyzes audio files and populates a MySQL database with raw audio feature data at various time resolutions. The feature data remains uninterpreted but is organized into streams depending on the type of processing employed (i.e. blind source separation, spectral analysis).
Symbolic
Isomer-Symbolic ingests symbolic (e.g. MIDI) files and populates a MySQL database with raw event data. The event data is organized into streams (separated by track) containing collections of raw NodeEvents.
Model (A.R.E.)
Isomer-Model normalizes raw analy­sis data (sym­bolic and/or audio) and populates a single (or multiple) MySQL database(s) with observed and interpreted feature representations. Multiple model abstractions of the same input may exist simultaneously in discrete databases, a key feature of Isomer that encourages the emergence of a desired representation schema for the musical model.
Query
Isomer-Query allows users to mine Isomer-generated model data for specific values and relevant trends. High dimensional queries (specified using time and stream/sub-stream ranges) can be made to model databases. Query results constitute the subset of model features available for subsequent processing.
Segment
Isomer-Segment detects perceptual changes over any subset of vectors in a given model. Once this new gestalt-map of the model is defined, Isomer-Segment searches for points of alignment between the various sub-streams (melody, texture, rhythm, harmony) and segments the model accordingly.

Model Processing

Similarity
To maximize usability of the representations, the system must be capable of flexibly comparing their traits to determine potential influence and overall relatedness. Isomer-Similarity performs this task by matching range limits or geometric contours of model data segments for any combination of vectors.
Classify
Isomer-Classify applies machine learning classifiers to expose relationships between groups of specific model features and trends found in the larger corpus or externally sourced data (such as human-generated meta-descriptions).
Transform
Isomer-Transform combines machine-learned feature trends with an extensive library of transform algorithms. Transformations can be applied across parameters and across any number of models, giving Isomer the ability to generate original musical material.
Render
Isomer-Render is responsible for orchestrating Isomer-Transform output and rendering performances. Today, final results are auditioned by humans. In future updates, Isomer-Render will employ a series of fitness functions to allow users to establish targeted output.
Keyword
Isomer-Keyword is a standalone module designed to collect, catalog and datamine human-generated descriptors. Its purpose is to provide a framework for researching the associative connections between the keywords themselves. From there, the content representation produced by Isomer can be analyzed to discover points of contact; where musical features closely associate with keyword descriptors.

 

This modular architecture makes Isomer ideal for a wide range of applications. Modules can be connected in myriad ways to create a wide range of workflows options.

Model-Based Music Generation

Isomer Music Generation

Trained Description Tagging

Isomer Tagging Process Flow

Reader Interactions

Leave a Reply

Your email address will not be published. Required fields are marked *