From Development to Production

Learn how to rapidly develop and deploy with FileMaker tools

FileMaker makes it very easy to rapidly develop applications and release them to users.  There is no code to integrate, no application binaries to generate, and no friction between development and release. What’s generated on the screen in FileMaker is immediately available to users on save. The process from Development to Production can be as simple as a single step. In reality, there are reasons to put controls around this process to ensure proper software quality assurance. Traditionally, software in production was managed by a different business group called Operations, probably to make sure that developers weren’t making changes to production that might bring negative side effects to the users.

DevOps is a model which integrates these two groups into a single support team for an application. The application team becomes involved with every aspect of the application, from development to release to production. This creates a cooperative environment for updating, testing, and releasing new features. The process begins with separating application development into three distinct environments; Development, Testing, and Production. Each environment provides controls, and a gateway for moving to the next stage.

Build, Test, Release

Imagine each environment as its own sandbox, with separate data and programming in each one. In the Development environment, new features are built, bugs are fixed, and programming is updated. Once the development is satisfactorily completed, the application can be moved to the Testing environment for quality controls, user acceptance testing, and feedback. After testing is passed by the application, it is ready for release.  At this point not every version needs to be released.  A decision can be made if the new features are complete enough or the fixed bugs are significant enough to move to production. A great tool for version controlling software releases is GitHub.

Making the move

There are many tools available for migrating code from one environment to the next, however FileMaker development, as a binary file, presents a different set of challenges for moving individual code changes. The primary method used has been to separate the data file from the interface file.  This allows updated interfaces to be copied to production without concern for data loss. Since most updates are delivered through the interface and not the data model, this process has allowed developers to release to production in a controlled process.  There are three methods currently available to FileMaker developers:

1. Overwrite production with the new development file
   • Production data is lost
2. Copy development changes to production file
   • Manual process increases risks of errors in production
3. Import production data into the development file
   • Manual process increases risks of production data loss

FM Data Migration Tool

Each of these methods has its strengths and weaknesses. With the recent release of the FM Data Migration Tool, Claris has made the third option much more robust and attractive for migrating development changes across environments. In fact, importing production data has been automated to the point that imports that took hours are accomplished in minutes. The Data Migration Tool is available with a Claris Developer subscription.

The tool works by analyzing the application file and copying the data tables in blocks.  The import process imports record by record, while the Data Migration Tool imports data table by table. The process is fast, effective, and automated.  Record IDs, internal unique record identifiers, are also preserved in the Migration process, unlike during the import process.  RecordIDs are key values used throughout XML and the Data API.

DevOps

A proper DevOps process requires a separation of Development and Production at minimum. Developers should not even have access to Production code, and all changes must originate in Development. This allows developers to continue managing and improving the application programming without impacting the production version that people are currently using. At some point in time, the project management of the application will determine when enough development work has been included to add value to the end user experience and is worth releasing.

Now, it becomes beneficial to move from the Development environment to a Test environment. This allows developers to continue working, without impacting the results of of quality assurance testing.  It is difficult to test an application that is continuing to introduce new changes. If bugs are found in testing, the file should go back to Development for updates, before moving back to production. If bugs are fixed directly in Test, then care must be taken to make the fixes in Development as well, otherwise the next version could reintroduce the same bug.

Once testing is completed, the application is ready to move to production.  Up until now, data migration has not been a concern, because Development and Testing usually use dummy data that doesn’t need to be moved from one environment to the next.  For the move to Production, all production data must be carefully preserved to avoid data loss. First, version the file to release, using local tools or tools provided by services like GitHub. Semantic Versioning is a way to distinguish the releases as major, minor, or patches.

Migrate

iCore provides a data migration assistant called Migrate that prepares the application file with production data ready for release. Migrate automates the usage of the FM Data Migration Tool, and integrates it with the FileMaker workflow. Migrate is a free FileMaker AddOn that can be installed in the development application, or used in a helper file for managing the migration to production.

Migration to production requires access to the Production Admin Console, a local application file security account with [Full Access] or the fmmigration extended privilege, and local access to the current production file and the new production file. The fmmigration account can be a local read only account with no fmapp or fmwebdirect access. The account does require All Available menu commands.

Automating the Migration

1. Prepare the new file for migration. Install the Migrate AddOn in the new file.  Point the tool to the location of the production file, allow the tool to create a clone of the new file, and select a target folder for the migrated file. Migrate will prompt to install or select the FM Data Migration Tool available through a Claris Developer Subscription.
2. Turn off the production file from the Admin Console. Run a backup of production while the file is off.
3. Run Migrate. There is a prompt for a username and password which must be in the new and production versions.  All data from the production file will be copied to the new file.  For files up to 5 GB, the process will be a few minutes or less. The migrated file will be located in the target folder. By default, the migrated file will maintain the security accounts from the production file. If developer and [Full Access] accounts are turned off in production, they will be turned off in the migrated file even if they were left on in Test.
4. Replace the production file with the migrated file created in the last step.  Turn the application back on from the Admin Console.  The new application version has now been released to production.

Utilizing Migrate in the DevOps process can maintain the benefits of Continuous Delivery while adding the controls of DevOps management. Previous release processes for FileMaker have either been onerous or tedious, leading many developers to avoid the procedures entirely and make changes directly to production, a risky process. Migrate automates much of the process, and allows a release to occur with as little as 5-10 minutes of production downtime. There is no manually copying programming or manually importing records, allowing release planning to flow in a more consistent and controlled manner.