benchFairy benchFairy - 3 months ago 11
JSON Question

Dynamically updating PHP curl_multi_init URL's based on items within folder directory

I am trying to call the information from an API to retrieve the data for items within the $baseURL.

$url1


is used to GET the availability json data.

$url2


is used to GET the json data for an individual item. e.g: Content, images, multimedia etc.

<?php

$APIKEY = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
$baseURL = 'http://feeds.example.com/v1/';
$ext = '.json';

// calls latest updated items
$url1 = 'folder';
// calls specific ID of item with corresponding details
$url2 = 'folder/item{id}';
$ch1 = curl_init($baseURL . $url1 . $ext);
$ch2 = curl_init($baseURL . $url2 . $ext);

$headers = array();
$headers[] = 'x-auth-token:' . $APIKEY;
$headers[] = 'X-Affiliate-Authentication:'. $APIKEY;
$headers[] = 'Content-Type: application/x-www-form-urlencoded; charset=utf-8';

curl_setopt($ch1, CURLOPT_USERPWD, $APIKEY);
curl_setopt($ch1, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, true);

curl_setopt($ch2, CURLOPT_USERPWD, $APIKEY);
curl_setopt($ch2, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch2, CURLOPT_RETURNTRANSFER, true);

$mh = curl_multi_init();

curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);

$active = null;

do {
curl_multi_exec($mh, $active);
} while ($active);

$response_1 = curl_multi_getcontent($ch1);
$response_2 = curl_multi_getcontent($ch2);
echo "$response_1 $response_2";
// Add function here

// Close function here
curl_multi_remove_handle($mh, $ch1);
curl_multi_remove_handle($mh, $ch2);

curl_multi_close($mh);

?>


'response_1' outputs (which is as expected):

{"data":[{"id":2,"last_update":"2016-08-26 15:43:39"},
{"id":6,"last_update":"2016-08-26 08:57:53"},
{"id":7,"last_update":"2016-08-23 16:49:15"},
{"id":8,"last_update":"2016-08-19 10:20:00"},
{"id":9,"last_update":"2016-08-28 23:44:25"},
{"id":28,"last_update":"2016-08-27 12:45:16"},
{"id":29,"last_update":"2016-08-12 07:55:32"},
{"id":30,"last_update":"2016-07-29 11:00:23"},
{"id":31,"last_update":"2016-07-05 09:48:50"}
.
..
...


'response_2' outputs (which is expected):

{"data":[
{
"id":2,"last_update":"2016-08-26 15:43:39",
"address":
{
"address1":"Foo Bar",
"address2":"Foo","city":"Bar","state":"FooBar","zip_code":"F00 84R","country":"Foo Bar",
},
"details":
{
"dwelling_name":"FooBar Hotel","hotel_type":"Big",
"maximum_capacity":16,
"base_capacity":16,
"bedrooms":6,
"bathrooms":3,
"currency":"GBP"
},
.
..
...


I need to determine the best method to dynamically update the $url[]. The following is my logic and won't work if you ran it because of course it is just my notes...

// folder/item{id}
$url[];
// Retrieved from 3rd party API for comparison
$storedUpdate;

foreach {
item{id}
if $storedUpdate and item{id} matches item{id} from $url1 {
do nothing
else {
POST item{id} data to 3rd party API
POST $latestUpdate to 3rd party API
}
};


I chose curl_multi_init due to its ability (from reading up on it) to parse data in a asynchronized manner. With item{ids} easily exceeding 3500, it seems illogical to manually code these into my work which is why I am looking at the plausibility of doing it dynamically.

I have been scouring the internet, forums and php curl manual to find a means of dynamically updating $url2 so that it GET's all data.

I believe I can use $url[] to input multiple urls, the problem seems to remain that I would therefore have to manually input these URLs.

feeds.example.com/v1/folder/{id}.json


I won't beat around the bush with this, I have learned this in a very short time frame and am at my whits end trying to determine the best practise for achieving my aim.

Any help would be much appreciated.

Answer

If I'm understanding you correctly, then your logic is slightly off. Your second request depends on information returned in the first request, therefore you require a synchronous flow not an asynchronous one. To do this asynchronously would require... time travel.

If you still want to reap the benefits of the curl_multi... functions, this will have to be limited to the second request(s). For example:

<?php

 $APIKEY = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
 $baseURL = 'http://feeds.example.com/v1/';
 $ext =  '.json';

 // calls latest updated items
 $url   = 'folder';

 $ch1 = curl_init($baseURL . $url . $ext);

 $headers = array();
 $headers[] = 'x-auth-token:' . $APIKEY;
 $headers[] = 'X-Affiliate-Authentication:'. $APIKEY;
 $headers[] = 'Content-Type: application/x-www-form-urlencoded; charset=utf-8';

 curl_setopt($ch1, CURLOPT_USERPWD, $APIKEY);
 curl_setopt($ch1, CURLOPT_HTTPHEADER, $headers);
 curl_setopt($ch1, CURLOPT_RETURNTRANSFER, true);

 $response_1 = curl_exec($ch1);
 $response_1_json = json_decode($response_1, true);
 $response_1_data = $response_1_json['data'];

 $mh = curl_multi_init();
 $multiHandles = [];

 foreach ($response_1_data as $key => $item) {
     $id = $item['id'];
     $multiHandles[$key] = curl_init($baseURL . 'folder/item' . $id . $ext);
     curl_setopt($multiHandles[$key], CURLOPT_USERPWD, $APIKEY);
     curl_setopt($multiHandles[$key], CURLOPT_HTTPHEADER, $headers);
     curl_setopt($multiHandles[$key], CURLOPT_RETURNTRANSFER, true);
     curl_multi_add_handle($mh, $multiHandles[$key]);
 }

 $active = null;

 do {
   curl_multi_exec($mh, $active);
 } while ($active);

 $response_2 = "";

 foreach ($response_1_data as $key => $item) {
     $response_2 .= curl_multi_getcontent($multiHandles[$key]);
     curl_multi_remove_handle($mh, $multiHandles[$key]);
 }

 echo $response_2;

 curl_multi_close($mh);

 ?>

I've not had chance to test this so it's likely something is off, but hopefully you get the gist of what I'm doing here. Hope this helps :)

Comments