Try to insert data from api using batch

I´m trying to get data from PokeAPI and insert in a Customo object and this is what i have so far.

public with sharing class PokemonBatch implements Database.Batchable<SObject>,Database.AllowsCallouts{

public Iterable<SObject> start(Database.BatchableContext context){
return Database.getQueryLocator('SELECT Id FROM Pokemon__c');
}
public void execute(Database.BatchableContext context,List<SObject> scope){

List<Pokemon__c> pokemonList = new List<Pokemon__c>();
for (Object obj : scope) {
Http http = new Http();
HttpRequest request = new HttpRequest();
request.setEndpoint('https://pokeapi.co/api/v2/pokemon?limit=905');
request.setMethod('GET');
HttpResponse response = http.send(request);
// Check if Connection was successful
if (response.getStatusCode() == 200) {
//deserialize response
Map<String,Object> responseData = (Map<String,Object>) JSON.deserializeUntyped(response.getBody());
List<Object> results = (List<Object>) responseData.get('results');
//get url of pokemon's endpoints
for (Object result : results) {
Map<String,Object> pokemonData = (Map<String,Object>) result;
String pokemonUrl = (String) pokemonData.get('url');
//Make call to new endpoint
HttpRequest detailsRequest = new HttpRequest();
detailsRequest.setEndpoint(pokemonUrl);
detailsRequest.setMethod('GET');
HttpResponse detailsResponse = new Http().send(detailsRequest);
// Check if Connection to the new endpoint was successful
if (detailsResponse.getStatusCode() == 200) {
//deserialize response
Map<String,Object> detailData = (Map<String,Object>) JSON.deserializeUntyped(detailsResponse.getBody());
// get fields from detail data
//***** get the id
Integer id = (Integer) detailData.get('id');
//***** Check if the id is greater than the number of pokemons we want to get
//***** we only want the pokemons until the 8th generation
if(id > 905){
return;
}
//***** get and convert height and weight to the correct units
Double height = (Double) detailData.get('height')/10;
Double weight = (Double) detailData.get('weight')/10;
//***** get the name
String name = (String) detailData.get('name');
//***** get and handle multiple types
List<Object> types = (List<Object>) detailData.get('types');
List<String> typeList = new List<String>();
for (Object type : types) {
Map<String,Object> typeData = (Map<String,Object>) type;
Map<String,Object> typeName = (Map<String,Object>) typeData.get('type');
String nameType = (String) typeName.get('name');
typeList.add(nameType);
}
//***** get species url to adquire the generation
Map<String,Object> species = (Map<String,Object>) detailData.get('species');
String speciesUrl = (String) species.get('url');
// make a call to the species endpoint
HttpRequest speciesRequest = new HttpRequest();
speciesRequest.setEndpoint(speciesUrl);
speciesRequest.setMethod('GET');
HttpResponse speciesResponse = new Http().send(speciesRequest);
// Check if Connection to the new endpoint was successful
if (speciesResponse.getStatusCode() == 200){
//deserialize response
Map<String,Object> speciesDetails = (Map<String,Object>) JSON.deserializeUntyped(speciesResponse.getBody());
//***** get the generation url and extract the the generation number from the end
Map<String,Object> generationDetails = (Map<String,Object>) speciesDetails.get('generation');
String generationUrl = (String) generationDetails.get('url');
String generation = generationUrl.substring(generationUrl.length() - 2, generationUrl.length() -1);
//***** get the sprites
Map<String,Object> sprites = (Map<String,Object>) detailData.get('sprites');
String spriteUrl = (String) sprites.get('front_default');
//***** create a new pokemon object and insert the data extratted fom the API
Pokemon__c pokemon = new Pokemon__c(Name=name.capitalize(),
PokeIndex__c=id,
Peso__c = String.valueOf(weight + ' kg'),
Altura__c = String.valueOf(height + ' mts'),
Tipo__c = String.join(typeList, ';'),
Geracao__c = Integer.valueOf(generation),
Foto_URL__c = spriteUrl
);
pokemonList.add(pokemon);
}
}
//***** insert list of records only if the list is not empty
}
}
}
if (!pokemonList.isEmpty()) {
insert pokemonList;
}
}
public void finish(Database.BatchableContext context){
// nothing
}
}

but the batch do not insert anything it shows completed but 0 batches processed. Any thoughts?

July 3, 2023
·
Answer
·
Like
0
·
Follow
0

Bryan Leaman 6
You're invoking a callout individually for each record in the query: SELECT Id FROM Pokemon__c. But you aren't including anything in the callout to identify the individual record you're calling out for.
for (Object obj : scope) { // there's a callout for each record selected // but nothing in the body of the batch references your loop variable "obj" }

A single Apex transaction can make a maximum of 100 callouts to an HTTP request or an API call.

How many records are in that object?
How large a batch size are you specifying when you launch the batch process?

Trying to determine what you're actually wanting to accomplish, it seems that maybe you're calling out to obtain a list of records you want to insert into your Pokemon__c data object.

How many pokemon can be returned by that callout? If it's under 2000, you can simply call-out, build a list and insert them all at once.

If you're expecting 20,000 (for example), then you would need to batch the insert statements, not a query over an empty object.

Could it be that what you really want is to perform the callout in your start method, build a list of pokemon__c records in the start method and return that list to the batch process, Then the execute method would simply perform the insert?

public with sharing class PokemonBatch implements Database.Batchable<SObject>,Database.AllowsCallouts{ public List<SObject> start(Database.BatchableContext context){ List<Pokemon__c> pokemonList = new List<Pokemon__c>(); Http http = new Http(); HttpRequest request = new HttpRequest(); request.setEndpoint('https://pokeapi.co/api/v2/pokemon?limit=905'); request.setMethod('GET'); HttpResponse response = http.send(request); // Check if Connection was successful if (response.getStatusCode() == 200) { //deserialize response Map<String,Object> responseData = (Map<String,Object>) JSON.deserializeUntyped(response.getBody()); List<Object> results = (List<Object>) responseData.get('results'); //get url of pokemon's endpoints for (Object result : results) { Map<String,Object> pokemonData = (Map<String,Object>) result; String pokemonUrl = (String) pokemonData.get('url'); //Make call to new endpoint HttpRequest detailsRequest = new HttpRequest(); detailsRequest.setEndpoint(pokemonUrl); detailsRequest.setMethod('GET'); HttpResponse detailsResponse = new Http().send(detailsRequest); // Check if Connection to the new endpoint was successful if (detailsResponse.getStatusCode() == 200) { //deserialize response Map<String,Object> detailData = (Map<String,Object>) JSON.deserializeUntyped(detailsResponse.getBody()); // get fields from detail data //***** get the id Integer id = (Integer) detailData.get('id'); //***** Check if the id is greater than the number of pokemons we want to get //***** we only want the pokemons until the 8th generation if(id > 905){ return; } //***** get and convert height and weight to the correct units Double height = (Double) detailData.get('height')/10; Double weight = (Double) detailData.get('weight')/10; //***** get the name String name = (String) detailData.get('name'); //***** get and handle multiple types List<Object> types = (List<Object>) detailData.get('types'); List<String> typeList = new List<String>(); for (Object type : types) { Map<String,Object> typeData = (Map<String,Object>) type; Map<String,Object> typeName = (Map<String,Object>) typeData.get('type'); String nameType = (String) typeName.get('name'); typeList.add(nameType); } //***** get species url to adquire the generation Map<String,Object> species = (Map<String,Object>) detailData.get('species'); String speciesUrl = (String) species.get('url'); // make a call to the species endpoint HttpRequest speciesRequest = new HttpRequest(); speciesRequest.setEndpoint(speciesUrl); speciesRequest.setMethod('GET'); HttpResponse speciesResponse = new Http().send(speciesRequest); // Check if Connection to the new endpoint was successful if (speciesResponse.getStatusCode() == 200){ //deserialize response Map<String,Object> speciesDetails = (Map<String,Object>) JSON.deserializeUntyped(speciesResponse.getBody()); //***** get the generation url and extract the the generation number from the end Map<String,Object> generationDetails = (Map<String,Object>) speciesDetails.get('generation'); String generationUrl = (String) generationDetails.get('url'); String generation = generationUrl.substring(generationUrl.length() - 2, generationUrl.length() -1); //***** get the sprites Map<String,Object> sprites = (Map<String,Object>) detailData.get('sprites'); String spriteUrl = (String) sprites.get('front_default'); //***** create a new pokemon object and insert the data extratted fom the API Pokemon__c pokemon = new Pokemon__c(Name=name.capitalize(), PokeIndex__c=id, Peso__c = String.valueOf(weight + ' kg'), Altura__c = String.valueOf(height + ' mts'), Tipo__c = String.join(typeList, ';'), Geracao__c = Integer.valueOf(generation), Foto_URL__c = spriteUrl ); pokemonList.add(pokemon); } } } } return pokemonList; } public void execute(Database.BatchableContext context,List<SObject> scope){ List<Pokemon__c> pokemonList = (List<Pokemon__c>) scope; if (!pokemonList.isEmpty()) { insert pokemonList; } } public void finish(Database.BatchableContext context){ // nothing } }

July 3, 2023
·
Like
0
·
Dislike
0

Paulo Proença
The problema is the i want to retrieve 905 pokemos from the APi and i try various aproaches but i´m allawys encounter CPU time limit exceeded or too many callouts issues because a need to do 905 requests to get each pokemon data and again 905 requests to get the generation.I allready had try create a list before call de execute but because the start isn't executesd in batchs i will have the same limit problems. I could do this using queueable jobs but because i can only channeling 5 i can not get all of them. I wonder if there is a way you can check if the queueable stack is max out and clear it and reuse it again, maintaining the sequence?
I already kind of solve the problem by using the scope has a loop allways getting the scope.get(0) a put it on the endpoint so it get one at a time but it removes the point of the batch because my batch has size 1, so i´m still looking for a cleaner and efficient solution.

July 4, 2023
·
Like
0
·
Dislike
0

Subrat (Salesforce Developers)
Hello ,

To resolve these issues, you need to update your code logic to ensure that you are processing the records from the scope list correctly and making the necessary API calls to retrieve the Pokemon data. You should iterate over the scope list and use each record to determine the specific Pokemon you need to fetch data for and then insert the relevant Pokemon records into the pokemonList before finally inserting them.

Here's an updated version of the execute method that should help you get started:
public void execute(Database.BatchableContext context, List<SObject> scope) { List<Pokemon_c> pokemonList = new List<Pokemon_c>(); for (SObject record : scope) { Pokemon_c pokemonRecord = (Pokemon_c) record; // Use the pokemonRecord to determine the specific Pokemon data to fetch // Make API calls and extract Pokemon data // Create a new Pokemon__c record and set the extracted data // Add the Pokemon__c record to the pokemonList } if (!pokemonList.isEmpty()) { insert pokemonList; } }
You'll need to update the code within the for loop to fetch the specific Pokemon data based on the pokemonRecord and create new Pokemon__c records with the extracted data. Ensure that you properly populate the pokemonList with the newly created records before inserting them.

Hope this helps !
Thank you.

July 5, 2023
·
Like
0
·
Dislike
0

Paulo Proença
Thanks Guys!! Your solutions made me think and i was not thinking right. All i have to do is make a call to the api to get a list of string with the name of pokemons in the start method and iterate it in the execute method and all done. Here is the code:

public class PopulatePokemonBatch implements Database.Batchable<String>, Database.AllowsCallouts {
public Iterable<String> start(Database.BatchableContext bc)
{
List<String> pokeList = new List<String>();
//Make API call
Http http = new Http();
HttpRequest request = new HttpRequest();
request.setEndpoint('https://pokeapi.co/api/v2/pokemon?limit=905');
request.setMethod('GET');
HttpResponse response = http.send(request);
// Check if Connection was successful
if (response.getStatusCode() == 200)
{
//deserialize response
Map<String,Object> responseData = (Map<String,Object>) JSON.deserializeUntyped(response.getBody());
List<Object> results = (List<Object>) responseData.get('results');
//get url of pokemon's endpoints
for (Object result : results)
{
Map<String,Object> pokemonData = (Map<String,Object>) result;
String pokeName = (String) pokemonData.get('name');
pokeList.add(pokeName);
}
}
return pokeList;
}

public void execute(Database.BatchableContext bc,List<String> scope)
{
List<Pokemon__c> pokemonList = new List<Pokemon__c>();
for(String record : scope){
String pokeName = record;
HttpRequest detailsRequest = new HttpRequest();
detailsRequest.setEndpoint('https://pokeapi.co/api/v2/pokemon/'+pokeName);
detailsRequest.setMethod('GET');
HttpResponse detailsResponse = new Http().send(detailsRequest);
// Check if Connection to the new endpoint was successful
if (detailsResponse.getStatusCode() == 200)
{
//deserialize response
Map<String,Object> detailData = (Map<String,Object>) JSON.deserializeUntyped(detailsResponse.getBody());
// get fields from detail data
//***** get the id
Integer id = (Integer) detailData.get('id');
//***** Check if the id is greater than the number of pokemons we want to get
//***** we only want the pokemons until the 8th generation

//***** get and convert height and weight to the correct units
Double height = (Double) detailData.get('height')/10;
Double weight = (Double) detailData.get('weight')/10;
//***** get the name
// String name = (String) detailData.get('name');
//***** get and handle multiple types
List<Object> types = (List<Object>) detailData.get('types');
List<String> typeList = new List<String>();
for (Object type : types)
{
Map<String,Object> typeData = (Map<String,Object>) type;
Map<String,Object> typeName = (Map<String,Object>) typeData.get('type');
String nameType = (String) typeName.get('name');
typeList.add(nameType);
}
//***** get species url to adquire the generation
Map<String,Object> species = (Map<String,Object>) detailData.get('species');
String speciesUrl = (String) species.get('url');
// make a call to the species endpoint
HttpRequest speciesRequest = new HttpRequest();
speciesRequest.setEndpoint(speciesUrl);
speciesRequest.setMethod('GET');
HttpResponse speciesResponse = new Http().send(speciesRequest);
// Check if Connection to the new endpoint was successful
if (speciesResponse.getStatusCode() == 200)
{
//deserialize response
Map<String,Object> speciesDetails = (Map<String,Object>) JSON.deserializeUntyped(speciesResponse.getBody());
//***** get the generation url and extract the the generation number from the end
Map<String,Object> generationDetails = (Map<String,Object>) speciesDetails.get('generation');
String generationUrl = (String) generationDetails.get('url');
String generation = generationUrl.substring(generationUrl.length() - 2, generationUrl.length() -1);
//***** get the sprites
Map<String,Object> sprites = (Map<String,Object>) detailData.get('sprites');
String spriteUrl = (String) sprites.get('front_default');
//***** create a new pokemon object and insert the data extratted fom the API
Pokemon__c pokemon = new Pokemon__c(Name=pokeName.capitalize(),
PokeIndex__c=id,
Peso__c = String.valueOf(weight + ' kg'),
Altura__c = String.valueOf(height + ' mts'),
Tipo__c = String.join(typeList, ';'),
Geracao__c = Integer.valueOf(generation),
Foto_URL__c = spriteUrl
);
pokemonList.add(pokemon);
}
}
}

if(!pokemonList.isEmpty()){
insert pokemonList;
}



}



public void finish(Database.BatchableContext bc)
{
system.debug('batch finished');
}
}

July 10, 2023
·
Like
0
·
Dislike
0

You need to sign in to do that.

Need an account? Sign Up

Have an account? Sign In

Dismiss

Browse by Topic

Welcome to Support!

Show

sorted by

Try to insert data from api using batch

You need to sign in to do that.