Print this page
In this topic
You can import documents into the application using flat files, or text-only load files. You can use the flat file import process with the following flat file types:
●Metadata flat file: DAT, CSV, or TXT files only.
●Image flat file: OPT or LFP files only. The image flat file links the metadata flat file to associated TIFF images.
When you import flat files, you can copy all associated full-text OCR and native source files to the application. When you add a flat file job, the application automatically saves all job settings and field mappings for later use.
Use the following guidelines to load non-native data into the application using the flat file import feature.
●Data load file (.DAT file): The first line in the load file contains the field names. Each subsequent line in the load file represents the metadata for a document. The following standard delimiters are supported :
oField delimiter: ( ¶ ) character 020
oText qualifier: ( þ) character 254
oNew line character: ( ® ) character 174
oMulti Value delimiter: ( ; ) semicolon
The application also accepts load files with a .TXT or .CSV extension.
●Date/time values: Always use separate fields for date and time values. For example, “Date Modified” and “Time Modified” must be separate fields, and always contain only the date and time, respectively.
●Image reference file (.OPT or .LFP): The application requires a standard format .OPT or .LFP file for loading of images. The image reference file contains path and document break information for images. The number of lines in the OPT file must match the number images provided. Always ensure the number associated with the first page of each document matches the value in a field for the corresponding row in the DAT file. Always name the first page of the document with Document ID and Control Number.
●Document text: Provide document text as separate text files. Use one text file per document and name each file using the Document ID and Control Number. A field in the DAT file (or other data load file) must reference the relative path to the text file. The application also accepts document text in a single field in the DAT file, but this is not recommended.
●Images: Document images must be copied to the exact path in the OPT file. Always verify that the number of lines in the OPT file matches the number of images.
●Native files: If native files are provided, the DAT file (or other data load file) must have a field with the relative path to the native file. Always name native files using the Document ID and Control Number of the document.
Use the following procedure to create a flat file import.
1.On the Case Home page, under Manage Documents, click Imports.
Note: As you complete each page in the Import window, the application saves the details. If you navigate away from the Import window, click Add to reopen the job. However, if you click Cancel, you lose all changes.
3.On the Import Details page, do the following:
a.In the Job type list, select Flat file.
b.Type a Name for the flat file import job.
c.Optionally, type a Description.
d.In the Date format list, select the default date format for the imported files.
Note: The application supports all date formats. The application uses the default date format only when a date is unclear, for example, 06/05/15.
4.On the Metadata File page, select the metadata file (.DAT, .CSV, .TXT) for import:
a.In the File repository list, select the file repository to import the data to. This list contains the file transfer repositories configured for your case.
b.In the Folder name list, select a folder.
c.In the File name list on the right, select the metadata file that you want to import.
5.On the Image File page, you can select an image file (.OPT or .LFP) for import.
a.Select Import image file.
b.In the Folder name list, select a folder, and in the File name list on the right, select the image file that you want to import.
6.On the Source Files page, use the following procedure to include source files with the import.
a.Select Copy source files to the server.
b.To overwrite matching source files, select Overwrite matching source and/or content files already on the server.
c.In the Folder name list, select a folder.
7.On the Delimiters page, set the delimiter character that separates field values in the flat file. You can also set the text qualifier that separates alphabetical characters from numeric characters in field values. After setting the delimiters, use this page to search for fields in the flat files. You can proceed to the next page only after the field search.
a.In the Column - fields list, select the delimiter character for columns and fields. For example, if the flat file is delimited by commas, select , (44).
b.In the Text qualifier list, select the text qualifier character, if required.
c.In the Row - documents list, do not change the default <CR>/<LF> value.
d.To search the flat file for available fields, click Get fields.
Note: Due to RPF server demands, this process can take a while. If you select an incorrect delimiter, an error message appears. If you change the import details or delimiters, click Get fields again to run the search again.
8.On the Map Files page, configure the location from which the application copies the associated native or full-text files.
a.If you are copying source files, in the Native path list, select the field to access native files. This option appears only when you select Copy source files to the server on the Source Files page.
Note: The selected field must appear in the relative path to the source files that you set on the Source Files page.
b.Under Full text, select one of the following options:
□None: Do not import full-text files.
□Metadata field: If you are importing text content in the flat file, select the field that contains this content. The application uses the selected field values to create associated text files for searching.
□File path: If you are importing full-text or OCR files, in the File path list, select the field that will reference the path to these files.
9.On the Existing Documents page, to determine how the application handles the import of existing documents, select one of the following Import action options, and then click Next.
Note: This page is similar to the Import documents feature. For more information, see Import documents.
Select one of the following Import action options:
oAbort import, if it contains an existing document ID
oImport new documents only: Data for documents that already exist in the case is ignored.
oUpdate existing documents only
oImport new documents and update existing documents
If you select Update existing documents only or Import new documents and update existing documents, additional options appear. See the following table for descriptions of the options.
For existing documents, update
●To update the listed fields, select Document Date, Description, Title, Type, or Source and attachments.
●To use null values to represent missing data when you update the selected fields, select Allow NULL updates (for fields above). For example, if a load file contains a document with a null value as its title, then the existing document is updated or overwritten with the null title.
●To update levels, select Levels.
●Do not update fields: Does not update data for existing documents. Only new documents contain import fields.
●Delete all field values. Do not update fields: Deletes all field values for documents in the import data set. When selecting this option, none of the updated documents will have import field values.
Note: This option only clears field values for documents in the import data set. For example, if a case contains existing documents A, B, and C, and the import data set contains documents C, D, and E, selecting Delete all field values does not affect documents A and B. It clears the metadata only for document C.
●Update only mapped fields: Deletes field values for documents in the case, and then adds those fields from the import data set to the case. This applies only to mapped fields. All other fields remain the same.
●Append only mapped fields: Either adds new field values for one-to-many fields or overwrites one-to-one fields for documents in the import data set. This only applies to mapped fields. All other fields remain the same.
●Do not update any parties: Does not update parties data in existing documents.
●Delete all party values. Do not update parties: Deletes all parties data values for documents in the import data set. When selecting this option, none of the updated documents will have parties data values.
●Update all mapped parties: Deletes the parties data values for documents in the import data set, and then adds new parties data values to the case. This option applies only to mapped parties data in the target data set.
●Append all mapped parties: Adds new mapped parties data values to documents and removes all parties information for updated documents.
●Do not update pages: Does not update pages data in documents.
●Delete all page values. Does not update pages: Deletes all pages data values for documents in the import data set. When selecting this option, none of the updated documents have pages data values.
●Update all pages: Deletes the pages data values for documents in the import data set, and then adds new pages data values to the case. This applies to both mapped pages data and all other pages data values in the target data set.
●Append all pages: Either overwrites new pages data fields in documents or overwrites fields for documents in the import data set.
10.On the Levels page, set the levels into which the application imports documents. By default, the maximum number of documents is 1,000 per level, but this is configurable. When the number of imported documents exceeds the set value, the application creates additional levels at the same depth, and appends each level with a number, for example, Level_0001, Level_0002. The application truncates level names that are longer than 20 characters.
a.To set the maximum number of documents per level, in the Max documents per level list, select a number.
b.Do one of the following:
□To select an existing level, in the Level list, select the root and sublevels. You must select at least one level for the document.
Tip: If there are many levels, filter the list by entering the first few letters of the level name.
□To create a new level, in the Level list, select Add new level, and then type the name of the level. The new level inherits the parent or root-level security settings.
Note: After you create a level, always synchronize the security groups. For information about syncing security groups, see Sync security.
□If the load file has a field with level name values, such as "Custodian," in the Level list, select Use data from load file, and then select the field.
11.On the Document ID page, set the document ID fields that the application uses to identify the imported files. You can select from the following options:
oExisting Document ID: Uses the existing document ID of a case.
The following options appear:
□Document ID: Selects the metadata field with the document ID.
□Image Document ID: Selects the metadata field that contains the value that links to the image load file. Usually, this field is Document ID.
□Source and Attachment relationship: When importing native source and attachment documents, select the load file that defines the relationships. If you are using a .DAT file, in the Source and Attachment relationship list, select one of these options. If there are no values in the selected field, the job still runs but reports an error.
♦LFP: Use the child designation from the LFP load file to create the source and attachment relationships.
♦Parent Doc ID: If this option is selected, the application expects attachment documents to contain the Document ID of the parent document in the selected field.
♦Attachment Doc IDs: If this option is selected, the application expects parent documents to contain the Document IDs of the attachment documents. If there are multiple attachments for one parent document, separate the Document IDs by a comma or a semi-colon.
♦Family Range: If this option is selected, the application expects all documents to contain a range of Document IDs, for example, DOC_0000001 - DOC_0000015. The first Document ID is the parent, and all other Document IDs between the first and last, including the last, are attachments. .
oGet Next - Levels Based: Uses the levels selected on the Levels page as the document ID. The application generates the document ID based on existing documents in the level.
If no documents exist, the application assigns the level names and a five-character
suffix as the ID number for the first document you upload. This number
increments as you upload subsequent documents.
For example, If you select Docs_1, Docs_2, and Docs_3 as levels, the application generates the document ID as Docs_1.Docs_2.Docs_3.00001, Docs_1.Docs_2.Docs_3.00002, and Docs_1.Docs_2.Docs_3.00003.
□To set the number of padding zeroes to use for the page number portion of the document ID of the imported documents, select a value in the Page length list.
□To view an example of how the document ID will appear, click View Document ID.
□For import jobs that include images, click Image Document ID and select the metadata field that contains the value that links to the image load file.
oGet Next - Sequential: Uses a sequential number as the document ID. The application generates the document ID based on existing documents in the level. If there are no documents, the application assigns a nine-character number (000000001) as the document ID.
□To create a custom document ID prefix, type the text into the Document ID prefix box. The application assigns a sequential number to the prefix. For example, if the prefix is AA, the application generates the document ID as AA000000001, AA000000002, and AA000000003.
□To view an example of how the document ID will appear, click View Document ID.
12.On the Field Map page, map the import fields to case fields in the application. If you are mapping one-to-many or many-to-one fields, you can select delimiters for the fields. Delimiters are not required for one-to-one fields. For each Import field, do the following:
oTo change the mapping of an import field, click a value in the Action column, and then select one of the following options:
□New: Creates a new field in the case database on import.
□Ignore: Excludes the field from the import.
□Import to: The application automatically shows you which case fields are valid for that import field based on field type and one-to-one or one-to-many relationships. For example, you can only map text fields to text fields, or Boolean fields to Boolean fields. One-to-many import fields can only map to one-to-many case fields.
oSelect a value in the Case field column that corresponds to the action selected for the import field.
oTo define characters to use as a delimiter, click the field in the Delimiter column an select a character.
oTo set a default value for a field in an imported document, click the field in the Default value column and then type a value.
14.On the Field Trumping Order page, you can determine the order in which the import fields will populate the application case field. Case fields affected by many-to-one mapping for the import job appear in the Case field pane on the left, if the case field is a one-to-one field.
Note: If the case field is a one-to-many field, values from all mapped import fields will be added to the case field.
i.To view and reorder the import fields, click the name of a case field. The order in which the import fields are used to populate the case field appears in the Rank column.
ii.To change ranking order, click on the handle of an import field row and drag it up or down.
16.On the Schedule page, set the start time for the import job, or place the job on hold to save as a draft job for later use. You can select from the following options:
oTo schedule a time, in the Start time box, select a date. In the second box, select a time.
oTo submit the job for processing immediately, leave the default Now setting.
oTo place a job on hold for later, select Place on hold, and then click Save as draft. You can return to a draft job at any time from the Imports page.
oTo schedule the job to run as a test, select Schedule as a test run only.
When you click Finish, the application adds the word TEST at the beginning of the description of the import on the Imports page.
The application returns to the Imports page, and indicates that your job is scheduled. The application saves your job definitions for processing. To view the import job details, click the name of the import job. The application displays the details in the Properties page.
When the flat file import job completes, it appears on the Imports page. Note that jobs saved as draft jobs also appear on the Imports page, but will not have run.
●To view the flat file job details, click the Name of the import job. The application displays the details on the Properties page.
●If the flat file import job completes with errors, view the errors on the Errors page.
●After the import job finishes, you can view the documents on the Documents page, in the List pane.