Today is: May 18, 2013
Core Data Migration - Standard Migration Part 2: Migration Boogaloo E-mail
Written by Mike Post   
Friday, 13 January 2012 16:11

UPDATE: Some people have said they've had mapping model problems. I personally didn't experience this, but the project is now uploaded to Github so you can check it out in it's entirety or modify it:
https://github.com/thepost/iPhoneCoreDataRecipesMigration

*********************************************************************************************************************************************************************

Please excuse the title, it's a personal rule of mine to append "boogaloo" to any subject that ends in the number 2.

"An in-depth discussion about standard migration is beyond the scope of this tutorial/book"

Remember this? This was the opening phrase in Part 1 of my Core Data Standard Migration tutorial, in which I:

  • gave a background on why the heck standard migration is necessary and rarely beyond any migration "scope";
  • explained the concepts behind core data migration and gave a state diagram in a way that Apple hasn't;
  • went on a rant in between all this, about the lack of detail that exists out there.

 

I've repeated the above quote in the introduction of Part 2 because it's worth reminding you that this IS NOT what this tutorial is about! We're going to dig in deep and tackle the hard problems of Standard Migration, not Lightweight Migration (which is extensively covered everywhere else, including the moon). The only reason there is even a Part 2 is because Part 1 was getting ridiculously long, and you would probably already be bored by the time the practical element was about to begin (MTV generation, low attention spans, etc).

I'm going to provide you with a logical style of learning the process. Firstly explaining what we need to do, then outlining the practical steps to achieve this. I'm also going to accompany the explanations with references to the Apple docs where I can, for verification if you need it explained in the more mechanical Apple-like manner, and so you can confirm that i'm not pulling this information from a mythical repository in the sky.

NOTE: The accompanying images and explanations of this tutorial are made with Xcode 4.2 in mind. You'll have to do things a little differently if you have any version of Xcode less than 4.0. Sorry, but I don't deal with backwards compatibility and neither should you, it's the Apple way.



ONWARDS & UPWARDS - ACHIEVING STANDARD MIGRATION:

1.
EXPLANATION:

  • We're going to be using a template project from the pre-existing Apple demo projects. Open Xcode and within the Documentation section in the Organizer window search for "coredatarecipes".
  • We want to make this as simple as possible and concentrate not so much on the UI, but the core data changes we're going to be making. So it makes sense to take a lot of the hassle out of it by disabling a lot of the pre-existing UI. For example, we're not using the "Edit" button or any functionality related to that in the tutorial, and leaving it in is prone to introducing bugs. Let's customise the Recipes app and make it stable for our purposes!

STEPS:

  • Open up the "iPhoneCoreDataRecipes" project from the Apple website or within Xcode itself. Take a snapshot of the application as an optional precaution.
  • It should have a Recipes.xcdatamodel which you are going to change. In the RecipeListTableViewController.m file, find the tableView:didSelectRowAtIndexPath: method and comment it out. - In the viewDidLoad method of the RecipeListTableViewController, comment out the lines that add the leftBarButtonItem and the rightBarButtonItem.
  • Build and run the Recipes app to make sure it's ok, then close again.

2.
EXPLANATION:

  • We need to add a Version hash in the same manner for Lightweight Migration, in order to check the model version upon opening of the persistent store.
  • We also need to add a Version Model Identifier to each of our models, 1.1 for the old version and 2.0 for the new version. This is so we can check out versions in code, specifically in this case, the persistentStoreCoordinator method.

STEPS:

  • Add a new Model Version. Select the .xcdatamodel file in the Project navigator. Then select Editor -> Add Model Version. Call it the EXACT same name as the existing schema, it will automatically get a number appended.
  • In Xcode 4, the newest xcdatamodel file created should be the model with the version number appended to it. This is now the default version.
  • If for any reason you want to explicitly select the xcdatamodel as the default version, in the Utilities view of the xcdatamodeld file, within the File Inspector column -> Versioned Core Data Model, select the current data model in the Current drop down menu. For the purpose of this exercise keep "Recipes 2" as the current model.
  • In the original xcdatamodel, add an explicit version number to the Version Model Identifier in the File Inspector of 1.1.
  • In the new xcdatamodel, add an explicit version number to the Version Model Identifier in the File Inspector of 2.0.

3.
EXPLANATION:

  • Now we're going to change the Data Model.

STEPS:

  • In the Recipe entity, change the "type" relationship to "to-many".
  • Add a "Chef" entity, with the attributes of "firstName" and "lastName", both optional and both strings.
  • Add a "recipes" relationship that is 1-to-many to the Recipe entity.
  • Add the corresponding "chef" relationship in the Recipe entity, with a many-to-1 relationship.

4.
EXPLANATION:

  • Alter the RecipesAppDelegate.m file's managedObjectModel method so that it doesn't automatically load an incorrect bundle (the alternative is to clean the project every time you create a new data model version).

STEPS:

  • Ensure that the following is commented out:
    //managedObjectModel = [[NSManagedObjectModel mergedModelFromBundles:nil] retain];
  • Instead explicitly find the bundle to merge the model with, using NSBundle's pathForResource:name ofType:extension method.

5.
EXPLANATION:

  • Reference: Pages 21-25 Core Data Migration Programming Guide
  • Ok so here we are approaching the crux of the migration, ahhh the split in the road between a Lightweight and Default Migration. You see in Lightweight migration, the transformations between the 2 different schemas are automatically inferred by Core Data. But this is more serious territory we're delving into here - we can't rely on it being inferred, we're going to have to provide that definition to Core Data ourselves!
  • The standard way to edit the Core Data model or schema has been in the Managed Object Model Editor. This is fine and necessary for defining the NEW schema. Now we need to define the link between the old and new. Guess what - there's an editor for that, the Mapping Model Editor! The mapping model editor allows you to customise the mappings between the Source and Destination entities and properties.
  • A Mapping Model is a collection of objects that specifies the transformations that are required, to migrate part of a store from one version of a model to another. A mapping model is directly related to the managed object classes in the Data Model …and a Model, Entity and Property all have their own corresponding mapping class (NSMappingModel, NSEntityMapping, NSPropertyMapping).

STEPS:

  • From the Xcode 4 File menu select "New" -> "New File" -> then select the "Mapping Model" icon from the "iOS" -> "Core Data" menu. Proceed to choose the Source and Destination models (in our case the Source model is the original and the Destination has 2 appended to it).

6.
EXPLANATION:

  • It's worth mentioning that you can take an optional extra step here of subclassing the NSEntityMigrationPolicy class. You only need to do this if you want to provide any custom code for the new entities you're creating, like validation, data verification, etc. Apple have a brief explanation of this on page 25 of the Core Data Migration PG.
  • NOTE: You do not NEED to subclass NSEntityMigrationPolicy to achieve Standard migration! One of the most common mis-truths out there is whether you need to subclass NSEntityMigrationPolicy. This is only if you want to provide further validation to any new properties, relationships, entities that you've set in your current model, and isn't taken care of already in your Mapping Model.

7.
EXPLANATION:

  • Initiating the Migration Process. The docs say that you can initiate migration in one of 3 ways:
    http://developer.apple.com/library/mac/#documentation/Cocoa/Conceptual/CoreDataVersioning/Articles/vmInitiating.html
  • For our example we're only interested in the method that supports versioning and the default migration process. Alternatively the 3rd option is confusing as to what Apple mean by the "Custom Version Skew Detection" method of migration, as they only give mention to it twice, and they don't specify how it's different to Default Migration… personally I want to investigate this more.
  • It's tempting and possible to just use addPersistentStoreWithType:configuration:URL:options:error: to automatically detect if versioning is needed. However, Apple states that this is costly. Therefore the preferred method is to first call NSManagedObjectModel's isConfiguration:compatibleWithStoreMetadata: method to see whether the store is compatible with managed object model.
  • In this step, check to see what version of the model we're using when the persistent store coordinator is allocated. If the model version is at least version 2 (for our example), then and only then check to see if migration has been performed.

STEPS:

  • Immediately after the Persistent Store Coordinator has been allocated in the persistentStoreCoordinator method, retrieve the set of version identifiers from our managed object model.
  • Add a conditional that checks for version 2.0 in the set, and created a boolean hasMigrated that is returned from checkForMigration method (yet to be implemented). Call it from the persistentStoreCoordinator method BETWEEN where you allocate the NSPersistentStoreCoordinator object and where you call addPersistentStoreWithType.
  • If hasMigrated is true then we have migrated in the current version, so reset the storePath string to point to the Recipes2 SQLite file.

8.
EXPLANATION:

  • Implement the checkForMigration method. This just checks to see if migration has already been performed, for the new version. It only gets called if we've detected that a new version is being used.
  • We'll need to retrieve the Source Metadata from the current NSManagedObjectModel, to see if it's compatible with the source store. We can do that with the isConfiguration: compatibleWithStoreMetadata: method.

STEPS:

  • Retrieve the Store source path for the SQLite file. If Recipes2 already exists, it doesn't necessarily mean that migration has been performed. It could mean that an attempted migration has been performed but failed! Other safeguards are needed in place.
  • Retrieve the Store URL for the path, and the subsequent Source Metadata dictionary for the store. - Check the configuration with the NSManagedObjectModel's method isConfiguration:compatibleWithStoreMetadata:
  • From here is what a lot of other tutorials don't do, so i'll help you understand the process along the way. Set up a breakpoint in the checkForMigration method, just underneath where the sourceMetadata is allocated.
  • Run the program, and let execution stop on your breakpoint.
  • Bring up your console and print the contents for the sourceMetadata object as such: po sourceMetadata
  • You'll notice that one of the objects in the dictionary is another dictionary called NSStoreModelVersionHashes. This represents your source data and should display all 4 of the original entities, in this case Image, Ingredient, Recipe, and RecipeType.
  • Step over to the line after your NSManagedObjectModel is allocated, and type the variable in the console: po destinationModel. This represents your destination model, and you can verify for this by looking out for your new Chef entity that should be printed to the console.
  • If you've followed all the steps correctly, isConfiguration:compatibleWithStoreMetadata: will return NO and the store won't be compatible with the model. This is where we call our performMigration method.

9.
EXPLANATION:

  • Create a method to perform the migration. I've named it as performMigrationWithSourceMetadata: toDestinationModel:
  • Perform the migration with an instance of NSMigrationManager. We need to retrieve the appropriate mapping model using our source and destination model instances.
  • The migration of the store is actually performed using migrateStoreFromURL. The Apple docs give an option of using one of 3 options - NSIgnorePersistentStoreVersioningOption, NSMigratePersistentStoresAutomaticallyOption, or NSInferMappingModelAutomaticallyOption. We do NOT want to infer the model here, that is for Lightweight Migration. We do not want to use the inferredMappingModelForSourceModel: destinationModel: error: method, nor do we want to to set a NSInferMappingModelAutomaticallyOption key in the PSC dictionary.

STEPS:

  • Instantiate a source model.
  • Instantiate a standard migration manager, with the source and destination models.
  • Instantiate a mapping model, also with the source and destination models.
  • Check that the mapping model is not nil, and execute the migrateStoreFromURL method, with the paths to the source and destination SQLite storage.

10.
EXPLANATION:

  • If all has gone well (and it should've with all the steps correctly followed), then checkForMigration should return YES. In either case, we'll need to reset the options parameter we're setting up the PSC with in the persistentStoreCoordinator method.

STEPS:

  • Now open the NSPersistentStore automatically with addPersistentStoreWithType:configuration:URL:options:error:
  • Add an entry to the options dictionary where the key is NSMigratePersistentStoresAutomaticallyOption and the value is an NSNumber object that represents YES (include an options Dictionary structure in the persistentStoreCoordinator method)!

That's it! Verify if the store data was compatible (it should be NO), and if the migration was successful (it should be YES) in the logs to put your mind at ease. Please ask any questions below if I've taken this newfound knowledge for granted and have left out any essential steps in this tutorial, or if I haven't explained anything well enough.

Below is my code for the whole RecipesAppDelegate implementation file. Just to recap, the main methods we've modified or added here are persistentStoreCoordinator, checkForMigration, and performMigrationWithSourceMetadata.


REFERENCES:

The much maligned official Apple docs:
http://developer.apple.com/library/mac/#documentation/Cocoa/Conceptual/CoreDataVersioning/Introduction/Introduction.html
These are 2 very good reads but are confined to OS X, not iOS:
http://www.timisted.net/blog/archive/core-data-migration/
http://pragprog.com/book/mzcd/core-data

 

RecipesAppDelegate.m:

#import "RecipesAppDelegate.h"
#import "RecipeListTableViewController.h"
#import "UnitConverterTableViewController.h"

@implementation RecipesAppDelegate

@synthesize window;
@synthesize tabBarController;
@synthesize recipeListController;


- (void)applicationDidFinishLaunching:(UIApplication *)application 
{
    recipeListController.managedObjectContext = self.managedObjectContext;
    
    [window addSubview:tabBarController.view];
    [window makeKeyAndVisible];
}


/**
 * applicationWillTerminate: saves changes in the application's managed object context,
 * before the application terminates.
 */
- (void)applicationWillTerminate:(UIApplication *)application {
	
    NSError *error;
    if (managedObjectContext != nil) {
        if ([managedObjectContext hasChanges] && ![managedObjectContext save:&error]) {
			NSLog(@"Unresolved error %@, %@", error, [error userInfo]);
			abort();
        } 
    }
}


#pragma mark -
#pragma mark Core Data stack

/**
 * Returns the managed object context for the application.
 * If the context doesn't already exist, 
 * it is created and bound to the persistent store coordinator for the application.
 */
- (NSManagedObjectContext *)managedObjectContext {
	
    if (managedObjectContext != nil) {
        return managedObjectContext;
    }
	
    NSPersistentStoreCoordinator *coordinator = [self persistentStoreCoordinator];
    if (coordinator != nil) {
        managedObjectContext = [NSManagedObjectContext new];
        [managedObjectContext setPersistentStoreCoordinator: coordinator];
    }
    return managedObjectContext;
}


/**
 * Returns the managed object model for the application.
 * If the model doesn't already exist, it is created by merging all the models found in the app bundle.
 */
- (NSManagedObjectModel *)managedObjectModel {
	
    if (managedObjectModel != nil) {
        return managedObjectModel;
    }

//    managedObjectModel = [[NSManagedObjectModel mergedModelFromBundles:nil] retain];    
    NSString *modelPath = [[NSBundle mainBundle] pathForResource:@"Recipes" ofType:@"momd"];
    NSURL *modelURL = [NSURL fileURLWithPath:modelPath];    
    managedObjectModel = [[NSManagedObjectModel alloc] initWithContentsOfURL:modelURL];        
    
    return managedObjectModel;
}


/**
 * Returns the persistent store coordinator for the application.
 * If the coordinator doesn't already exist, it is created and the application's store added to it.
 */
- (NSPersistentStoreCoordinator *)persistentStoreCoordinator {
	
    if (persistentStoreCoordinator != nil) {
        return persistentStoreCoordinator;
    }
		
	NSString *storePath = [[self applicationDocumentsDirectory] 
    						stringByAppendingPathComponent:@"Recipes.sqlite"];
    [storePath retain];
    
	NSFileManager *fileManager = [NSFileManager defaultManager];
	// If the expected store doesn't exist, copy the default store.
	if (![fileManager fileExistsAtPath:storePath]) {
		NSString *defaultStorePath = [[NSBundle mainBundle] pathForResource:@"Recipes" ofType:@"sqlite"];
		if (defaultStorePath) {
			[fileManager copyItemAtPath:defaultStorePath toPath:storePath error:NULL];
		}
	}
	
    persistentStoreCoordinator = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel: [self managedObjectModel]];
    
    //Check to see what version of the current model we're in. If it's >= 2.0, 
    //then and ONLY then check if migration has been performed...
    NSSet *versionIdentifiers = [[self managedObjectModel] versionIdentifiers];
    NSLog(@"Which Current Version is our .xcdatamodeld file set to? %@", versionIdentifiers);
    
    if ([versionIdentifiers containsObject:@"2.0"]) 
    {        
        BOOL hasMigrated = [self checkForMigration];
        
        if (hasMigrated==YES) {
            [storePath release];
            storePath = nil;
            storePath = [[self applicationDocumentsDirectory] 
            				stringByAppendingPathComponent:@"Recipes2.sqlite"];
        }
    }    
    
    NSURL *storeUrl = [NSURL fileURLWithPath:storePath];	
	NSError *error;    
    NSDictionary *pscOptions = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithBool:YES], NSMigratePersistentStoresAutomaticallyOption,
                                [NSNumber numberWithBool:NO], NSInferMappingModelAutomaticallyOption,
                                nil];
    
    if (![persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType 
                                                  configuration:nil 
                                                            URL:storeUrl 
                                                        options:pscOptions 
                                                          error:&error]) {
		NSLog(@"Unresolved error %@, %@", error, [error userInfo]);
		abort();
    }    
		
    return persistentStoreCoordinator;
}


- (BOOL)checkForMigration
{
    BOOL migrationSuccess = NO;
    NSString *storeSourcePath = [[self applicationDocumentsDirectory] 
    								stringByAppendingPathComponent:@"Recipes2.sqlite"];
    NSFileManager *fileManager = [NSFileManager defaultManager];
    
	if (![fileManager fileExistsAtPath:storeSourcePath]) {
        //Version 2 SQL has not been created yet, so the source is still version 1...
        storeSourcePath = [[self applicationDocumentsDirectory] 
        					stringByAppendingPathComponent:@"Recipes.sqlite"];
    }
	
    NSURL *storeSourceUrl = [NSURL fileURLWithPath: storeSourcePath];
	NSError *error = nil;        
    NSDictionary *sourceMetadata = [NSPersistentStoreCoordinator 
    									metadataForPersistentStoreOfType:NSSQLiteStoreType 
																	URL:storeSourceUrl 
																	error:&error];
    if (sourceMetadata) {
        NSString *configuration = nil;
        NSManagedObjectModel *destinationModel = [self.persistentStoreCoordinator managedObjectModel];
        
        //Our Source 1 is going to be incompatible with the Version 2 Model, our Source 2 won't be...
        BOOL pscCompatible = [destinationModel isConfiguration:configuration compatibleWithStoreMetadata:sourceMetadata];
        NSLog(@"Is the STORE data COMPATIBLE? %@", (pscCompatible==YES) ?@"YES" :@"NO");
                
        if (pscCompatible == NO) {
            migrationSuccess = [self performMigrationWithSourceMetadata:sourceMetadata toDestinationModel:destinationModel];
        }
    }
    else {
        NSLog(@"checkForMigration FAIL - No Source Metadata! \nERROR: %@", [error localizedDescription]);
    }
    return migrationSuccess;
}


- (BOOL)performMigrationWithSourceMetadata :(NSDictionary *)sourceMetadata 
toDestinationModel:(NSManagedObjectModel *)destinationModel
{
    BOOL migrationSuccess = NO;
    //Initialise a Migration Manager...
    NSManagedObjectModel *sourceModel = [NSManagedObjectModel mergedModelFromBundles:nil 
                                                                    forStoreMetadata:sourceMetadata];
    //Perform the migration...
    if (sourceModel) {
        NSMigrationManager *standardMigrationManager = [[NSMigrationManager alloc] 
        													initWithSourceModel:sourceModel 
                                                               destinationModel:destinationModel];
        //Retrieve the appropriate mapping model...
        NSMappingModel *mappingModel = [NSMappingModel mappingModelFromBundles:nil 
                                                                forSourceModel:sourceModel 
                                                              destinationModel:destinationModel];
        if (mappingModel) {
            NSError *error = nil;
            NSString *storeSourcePath = [[self applicationDocumentsDirectory] stringByAppendingPathComponent:@"Recipes.sqlite"];
            NSURL *storeSourceUrl = [NSURL fileURLWithPath: storeSourcePath];
            NSString *storeDestPath = [[self applicationDocumentsDirectory] stringByAppendingPathComponent:@"Recipes2.sqlite"];
            NSURL *storeDestUrl = [NSURL fileURLWithPath:storeDestPath];
            
            //Pass nil here because we don't want to use any of these options:
            //NSIgnorePersistentStoreVersioningOption, NSMigratePersistentStoresAutomaticallyOption, or NSInferMappingModelAutomaticallyOption
            NSDictionary *sourceStoreOptions = nil;
            NSDictionary *destinationStoreOptions = nil;
            
            migrationSuccess = [standardMigrationManager migrateStoreFromURL:storeSourceUrl 
                                                                        type:NSSQLiteStoreType
                                                                     options:sourceStoreOptions 
                                                            withMappingModel:mappingModel 
                                                            toDestinationURL:storeDestUrl 
                                                             destinationType:NSSQLiteStoreType 
                                                          destinationOptions:destinationStoreOptions 
                                                                       error:&error];
            NSLog(@"MIGRATION SUCCESSFUL? %@", (migrationSuccess==YES)?@"YES":@"NO");
        }
    }   
    else {
        //TODO: Error to user...
        NSLog(@"checkForMigration FAIL - No Mapping Model found!");
        abort();    
    }
    return migrationSuccess;
}//END


#pragma mark -
#pragma mark Application's documents directory

/**
 * Returns the path to the application's documents directory.
 */
- (NSString *)applicationDocumentsDirectory {
	return [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) lastObject];
}


#pragma mark -
#pragma mark Memory management

- (void)dealloc {
    [managedObjectContext release];
    [managedObjectModel release];
    [persistentStoreCoordinator release];
    
    [recipeListController release];
    [tabBarController release];
    [window release];
    [super dealloc];
}

@end