Rails flat file importation process automation. Flat lady automates the following processes :

1. Map the columns of any given flat file to a denormalized schema understood by your system
2. Validate the integrity of the imported data
3. Run the importation process in the background when asked by the user

it ain’t over until the flat_lady signs

In config/environment.rb:

config.gem ‘paperclip’
config.gem ‘state_machine’
config.gem ‘ar-extensions’
config.gem ‘fastercsv’
config.gem ‘haml’,
:version => ‘~> 2.0.4’

plugins :

./script/plugin install git://github.com/giraffesoft/resource_controller.git

now that the dependencies are installed, install the plugin:

Make sure the development database exists and run the generator:

A number of files will be created and instructions will be printed.

You may already have some of these files. Don’t worry. You’ll be asked if you want to overwrite them.

Run the migration:

Flat Lady uses the delayed_jobs plugin to run the importation related jobs in the background. So if it’s not installed already, you’ll have to install the plugin :

Make sure the development database exists and run the generator:

./script/generate flat_lady_background

for more information on the delayed_job usage, please refer to : http://github.com/tobi/delayed_job/tree/master.

Flat lady enables you to provide an importation interface for any model of your system.

The first step is to define a denormalized schema which will be used as a buffer to validate the data before the importation.

the second step is to define the “check_integrity” and “push!” methods of the generated model

Example :

let’s say you want to import contacts from many different csv files, but, in your app, contacts are define as follow :

you would start by defining a denormalized (flat) schema for contacts with many phone numbers :

this will create the scaffold with the required migrations and routes for the ImportedContact model.

Make sure you define the 2 following methods on the ImportedContact class

# this method will be called on each imported_contact directly after translating the csv file. flat_lady adds 2 integrity flags by
#default to the ImportedContact model. Those flags are : invalid_import and duplicated_import. Those flag should be set in the
# check_integrity method.

start the background worker

The users of your application should now be able to import contacts via /flat_lady/import_files/new

As your app evolves, you want to know that authentication still works. FlatLady’s opinion is that you should test its integration with your app using Cucumber.

In config/environments/test.rb:

Vendor the gems:

Don’t vendor nokogiri (due to its native extensions):

Run the Cucumber generator (if you haven’t already) and FlatLady’s feature generator:

David Fugère, Benjamin Thouret.

Ask the mailing list_