Importing Customers and Vehicles (Generic)

Summary

Torqueflow’s import system is designed around “upload and forget”. You drag one or more CSV or Excel files onto the page, and the system auto-detects what each file contains from the header row, groups them into a single import run, and processes them in the correct order. Password-protected Techman Excel files are supported natively. This article covers the generic flow; for the Techman-specific quirks (customer groups, UK dates, notification JSON) see data-import/importing-from-techman.

Prerequisites

• You must be signed in as an owner or admin with the settings.import.manage capability.
• Your data is in CSV or .xlsx format. Legacy .xls files are not supported - convert them first.

Steps

Prepare your files

1. Export customers and/or vehicles from your current system as CSV. Any combination works:
   • Single CSV with customers only.
   • Single CSV with customers and vehicles interleaved.
   • Multiple CSVs, one per entity type.
   • Mix of CSVs and a password-protected Techman Excel file.
2. Ensure the header row is present and uses recognisable field names. The auto-detection engine reads headers to identify the file - if a header is missing or mis-spelled, detection may fail.
3. You do not need to know or declare what each file is. Torqueflow figures it out.

Upload one or more files

1. Go to Settings > Import Data.
2. Drag your files onto the upload area, or click Select Files and pick them.
3. Any combination of .csv and .xlsx is allowed in a single batch.
4. The upload creates a single ImportRun record that groups all the files together.

Enter the Excel password (if applicable)

1. If you uploaded a password-protected .xlsx file (the format Techman typically sends), Torqueflow shows a password input labelled Enter the password from Techman’s second email.
2. Enter the password and click Decrypt. The password is never stored, logged, or persisted - it is used in-memory on the server only.
3. On successful decryption, each worksheet (tab) of the Excel file is extracted and treated as a separate CSV. A single Excel file with 11 tabs becomes 11 detected files in the run.
4. If the password is wrong, you see “Could not open the file - please check the password and try again.” The file stays in the upload - you only need to retype the password.
5. If the file is a non-encrypted .xlsx, Torqueflow reads it without a password.

Review the detection summary

1. After upload and decryption, the detection summary shows every file with:
   • Filename (or tab name if extracted from Excel).
   • Detected type - what the auto-detector thinks it is (e.g. “Techman CustomersAndVehicles”, “Customers only”, “Vehicles only”, “Products”, “Jobs”).
   • Row count.
   • Confidence indicator.
2. Unknown files are flagged with a warning: “This file wasn’t recognised. Check the format and try again.” Unknown files do not block the rest of the run.
3. A summary line at the top reads something like “Found 11 files: 9,408 customer-vehicle rows, 7,339 jobs, 3,667 products…”.

Review the cleanup preview

1. Click through to each detected file to see its preview.
2. The preview shows the first 100 records with cleanup applied:
   • Phone numbers normalised to E.164.
   • Dates converted to ISO 8601.
   • Boolean strings converted to real booleans.
   • Vehicle registration marks (VRM) cleaned up.
3. Flagged rows show warnings or errors. Fix them at source if there are patterns.

Handle duplicates

1. For each incoming customer or vehicle that matches an existing Torqueflow record (by name + phone, email, or VRM), you see a duplicate flag with options: Skip, Merge, or Create new.
2. Choose a default for the whole run via the Apply to all duplicates button, or handle each one individually.
3. Merge updates the existing record with the incoming data without overwriting existing non-null values.

Understand reconciliation via ImportMapping

1. Every imported record gets an ImportMapping entry that records where it came from: source_system, source_entity_type, source_id, and the new Torqueflow ID.
2. This means future imports can look up “is this record the same as one we already imported?” regardless of whether the earlier import was Techman, a CSV, or an Excel file.
3. ImportMapping has a unique constraint on (organization_id, source_system, source_entity_type, source_id) so you cannot accidentally map two Torqueflow records to the same external ID.

Trigger the import

1. When you are happy with the preview and duplicate choices, click Run Import.
2. The run status moves from ready to importing. Each file becomes its own background job, processed in batches of 100.
3. A progress bar shows “X of Y imported” per file, updated within 2 seconds of each batch completing.
4. Files are processed in the correct order - customers before the vehicles that reference them, for example.
5. You can navigate away and come back at any time. Progress is stored on the ImportRun and resumes on refresh.

Review the summary

1. When the run finishes, the summary shows per-file and per-run totals:
   • Total imported.
   • Skipped (duplicates).
   • Skipped (user choice).
   • Failed, with per-row reasons.
2. Download the error log CSV for anything that failed.
3. An audit log entry is created for the full run.

Expected Outcome

• You dragged files onto the upload area without needing to know what was in each one.
• Torqueflow detected, grouped, and processed them in the right order.
• Customers and vehicles are in your database with provenance tracked via ImportMapping.
• Excel files with passwords are handled natively - no manual tab-to-CSV conversion needed.
• Duplicates are resolved explicitly so no phantom records are created.

Troubleshooting

Problem: One of my files is flagged as “unrecognized”. Cause: The header row does not match any known signature. Possible causes: missing columns, renamed columns, extra whitespace or BOM characters in the header. Fix: Open the file in a spreadsheet, check the header row matches what the source system normally produces. Fix and re-upload. If the file is a custom export, you may need the Techman adapter or a different import method - contact support.

Problem: Excel decryption says password is correct but no tabs appear. Cause: The Excel file has tabs but they are all empty, or the header row is missing from each tab. Fix: Open the file in Excel and verify each tab has a header row and data. Re-save and re-upload.

Problem: I uploaded 5 files but only 4 appear in the detection summary. Cause: One of the files was empty, corrupted, or in an unsupported format. Fix: Check the file size and format. Re-export from the source system if needed.

Problem: Mixed CSV and Excel upload - duplicate detection seems wrong. Cause: If the same customer appears in both an Excel tab and a standalone CSV, both show up as separate ImportJobs and duplicate detection runs at the data level during import. Fix: This is expected. Let the import run - duplicate detection will handle it. Pick Merge as your default duplicate action if you expect overlap.

Problem: Phone numbers or VRMs are matching as duplicates when they should not. Cause: Normalisation can collapse values that look different but are actually the same. For VRMs, spaces and case are removed. Fix: Review flagged duplicates in the preview and use Create new for any that are genuinely distinct.

Problem: The password input never appears for my .xlsx file. Cause: The file is not actually password protected, or the system is reading it as unencrypted. Fix: Torqueflow reads unencrypted .xlsx files without a password - this is fine. If you believe the file should be encrypted, check the file properties in Excel.

Problem: Detection says “Techman UI export” but my data is from another system. Cause: Some systems use Techman-compatible exports (the header signature matches). Fix: This usually works fine - the Techman adapter will apply Techman cleanup rules. If your data has quirks that differ from Techman, review the preview carefully or contact support for a dedicated adapter.

Notes

• Passwords are handled in-memory only on the server. They are never written to the database, logs, or client storage. The decrypted workbook buffer is released immediately after extraction.
• Excel files larger than the configured limit are rejected with a clear size error before decryption is attempted. Contact support if your file exceeds the limit.
• CSV is a first-class format and will always be supported. Excel is additive.
• The auto-detection engine is built on header signatures. Adding a new source system is a matter of adding a new signature - contact support if you need support for a system that is not currently detected.
• Each ImportRun can be reviewed, rolled back, or exported for audit. See data-import/reviewing-and-rolling-back-an-import.

Permissions

• settings.import.manage - required to upload files and run imports. Assigned to owners and admins by default.