Sign Up ›

Browse by Topic
ShowAll Questionssorted byDate Posted
+ Start a Discussion
Jeffrey ZhangJeffrey Zhang 

Need some general help on csv's and java.

Not sure if this is the right section to ask, but I've been tasked with rebuilding an old java application that used SOAP calls for upserting data. The problem is the size of our org often lead to time out issues which is a pain with this app being run on a windows scheduler.

Was thinking of bulk api is a good solution for the upsert, the question is though, haven't really used java in a decade or so and not familiar with the options out there for CSV generation off off of like a map or other list options?  

Anyone have experience developing apps that dl data from salesforce, repackage and bulk api upsert back?

thanks!
Alain CabonAlain Cabon
Hello Jeffrey,

I am also a java developer (becoming an Apex developer) and I solved this problem using the powerful parser of Univocity.  http://www.univocity.com/pages/parsers-download

 https://github.com/uniVocity/univocity-parsers

https://github.com/uniVocity/csv-parsers-comparison

The common mistake of beginners is to use the simple split method of String for getting the CSV fields.
You need a real parser for CSV files and this parser is excellent (Detection Enabled everywhere from the header and content).
The sample concise code below will reformat of the exported Account.csv according the fields of the array (cols).

All the work is prepared by the CsvParserSettings and the CsvWriterSettings. 
package Salesforce;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;

import com.univocity.parsers.csv.CsvFormat;
import com.univocity.parsers.csv.CsvParser;
import com.univocity.parsers.csv.CsvParserSettings;
import com.univocity.parsers.csv.CsvWriter;
import com.univocity.parsers.csv.CsvWriterSettings;

public class ReadExport {

	static String str;

	static String[] cols = new String[] { "Id", "Name", "Type", "RecordTypeId",
			"ParentId", "Website", "NumberOfEmployees", "OwnerId",
			"CreatedDate", "CreatedById", "LastModifiedDate",
			"LastModifiedById", "SystemModstamp" };

	static PrintWriter out1;
	static {
		try {
			out1 = new PrintWriter(new FileWriter("C:\\tmp\\Account_reformatted.txt"));
		} catch (IOException e1) {
			e1.printStackTrace();
		}
	}

	public static void main(String[] args) {
		File file = new File("C:\\tmp\\Account.csv");
		treat(file);
	}

	static void treat(File file) {
		CsvParserSettings settings = new CsvParserSettings();
		settings.setHeaderExtractionEnabled(true);
		settings.setLineSeparatorDetectionEnabled(true);
		settings.setQuoteDetectionEnabled(true);
		settings.setSkipEmptyLines(true);
		settings.setDelimiterDetectionEnabled(true);
		settings.setMaxCharsPerColumn(10000);
		settings.selectFields(cols);

		CsvWriterSettings settings2 = new CsvWriterSettings();
		settings2.setQuoteAllFields(true);
		CsvFormat format = new CsvFormat();
		format.setDelimiter(',');
		settings2.setFormat(format);
		settings2.setHeaders(cols);

		CsvWriter csvw = new CsvWriter(out1, settings2);
		CsvParser parser = new CsvParser(settings);
		BufferedReader in = null;
		try {
			in = new BufferedReader(new FileReader(file));
			parser.beginParsing(in);
			System.out.println("--- BEGIN ---");
			csvw.writeHeaders();
			String[] row;
			while ((row = parser.parseNext()) != null) {
				csvw.writeRow(row);
			}
			System.out.println("-- END---");
			out1.close();
            parser.stopParsing();

		} catch (FileNotFoundException e) {
			e.printStackTrace();
		}		
	}
}

For filtering:

int recordtypeid = parser.getRecordMetadata().indexOf("RecordTypeId");
if (row[recordtypeid ] != null) { ... }

Regards

Alain
Jeffrey ZhangJeffrey Zhang
awesome. thanks! will try this.
Alain CabonAlain Cabon
Ok I hope that could help you.

Best regards

Alain