/v2/items Enabled
Pat Cavit.9234:
Hey folks, we’ve just enabled /v2/items. Documentation isn’t quite ready yet but I’ll be talking to poke and we’ll go from there.
Querying it is identical to the other /v2 APIs.
https://api.guildwars2.com/v2/items/12138
https://api.guildwars2.com/v2/items?page=0
https://api.guildwars2.com/v2/items?page=1&page_size=200
https://api.guildwars2.com/v2/items?ids=12138,57
smiley.1438:
Whoa, finally! Thanks!
(so you did this because you noticed me hammering the v1/items endpoint right now?
)
StevenL.3761:
It’s so fast. :O
Seems like a number of other bugs were fixed as well. Nice work!
poke.3712:
Documentation is now available on the wiki. For now, I’ll refer you to the v1 documentation for the type-specific properties. I’ll update that as soon as I verified whether the format changed for anything in v2.
smiley.1438:
Ok, one bug (or feature?):
When you send a chunk of ids and one of them is invalid, the API returns a 400 for the whole request. I’d prefer if it would return the available ones instead.
The invalid ones (i had in my DB still):
39925
40615
40898
40914
40922
40930
40978
43948
43949
StevenL.3761:
I’m PRETTY sure this looks wrong: https://api.guildwars2.com/v2/items/21092
Shouldn’t that ‘upgrade_component’ property instead be a ‘details’ property?
Pat Cavit.9234:
I’m PRETTY sure this looks wrong: https://api.guildwars2.com/v2/items/21092
Shouldn’t that ‘upgrade_component’ property instead be a ‘details’ property?
Hmm, looks like our tests weren’t checking for that. Probably just a mapping update, but we’ll take a look.
Pat Cavit.9234:
Documentation is now available on the wiki. For now, I’ll refer you to the v1 documentation for the type-specific properties. I’ll update that as soon as I verified whether the format changed for anything in v2.
Thanks a ton poke, sorry about the surprise launch. trying to get /v2 APIs out as fast as we finish cleaning them/making them scale, so this probably isn’t the last surprise :-\
darthmaim.6017:
Awesome! I love how everything gets done now so fast and that you are active on the forum!
Kasoki.7013:
Is there still no way to search for items by name?
The only way to do this (at least as far as I know) is to ask the server about every single item like this: http://jsfiddle.net/1zkftj7y/1/
poke.3712:
The full documentation is now available, including all edge cases and oddities currently in the API.
The only way to do this (at least as far as I know) is to ask the server about every single item like this: http://jsfiddle.net/1zkftj7y/1/
For a quick search, you should probably cache the item names on your end for now. Requesting all items for every single search seems like a very bad idea. If you want some data to start with, here are all item names in the API at this moment.
Also, it’s a very bad idea to fetch every single item at once. If you need multiple resources, fetch for multiple at once using the `ids` parameter. That reduces the requests you need to do by a lot and results in a much better performance (for both your client as well as the API server—HTTP requests are expensive!).
And if you don’t actually know which item ids you are looking for, instead of fetching all ids and then querying the information for all those ids, just use paging on the resource directly.
smiley.1438:
Is there still no way to search for items by name?
No, unfortunately not. That’s what we were discussing over here: https://forum-en.guildwars2.com/forum/community/api/v2-item-details-and-recipe-details/4434782
Kasoki.7013:
The full documentation is now available, including all edge cases and oddities currently in the API.
The only way to do this (at least as far as I know) is to ask the server about every single item like this: http://jsfiddle.net/1zkftj7y/1/
For a quick search, you should probably cache the item names on your end for now. Requesting all items for every single search seems like a very bad idea. If you want some data to start with, here are all item names in the API at this moment.
Also, it’s a very bad idea to fetch every single item at once. If you need multiple resources, fetch for multiple at once using the `ids` parameter. That reduces the requests you need to do by a lot and results in a much better performance (for both your client as well as the API server—HTTP requests are expensive!).
And if you don’t actually know which item ids you are looking for, instead of fetching all ids and then querying the information for all those ids, just use paging on the resource directly.
I’ve made this naive approach exactly to show how inconvenient it is to retrieve certain items at the moment.
The current API design somewhat encourages lots of API calls which are obviously bad both for Arena.NET and for the user. An item search API or an interface like your item_names.json would be quite a pleasure.
I’m currently caching all items in a similar structure as your item_names.json but to build this I still need to download every single item once. This is a pretty huge amount of data just to make a simple search: ~38k items = ~38k HTTP calls…
Pat Cavit.9234:
I’ve made this naive approach exactly to show how inconvenient it is to retrieve certain items at the moment.
The current API design somewhat encourages lots of API calls which are obviously bad both for Arena.NET and for the user. An item search API or an interface like your item_names.json would be quite a pleasure.
I’m currently caching all items in a similar structure as your item_names.json but to build this I still need to download every single item once. This is a pretty huge amount of data just to make a simple search: ~38k items = ~38k HTTP calls…
38000 items / 200 per page using pagination = 190 requests.
But yes, I agree a search API would be nice!
Pat Cavit.9234:
The full documentation is now available, including all edge cases and oddities currently in the API.
This is awesome, thanks so much poke! We’ll get the upgrade_components & default_skin bugs sorted pretty soon here.
Pat Cavit.9234:
https://api.guildwars2.com/v2/items/56
default_skin is now a number, like it should’ve been all along
https://api.guildwars2.com/v2/items/21092
UpgradeComponents now have a proper details object, like they should’ve had all along.
Sorry about the bugs! We have lots of items, some are weird XD
smiley.1438:
Thanks for the quick fixes!
How about the requests returning a 400 on single invalid item ids? Would be cool if the API would return a HTTP/206 Partial Content then and just send the data – would make our lives easier
Khisanth.2948:
A search API might be nice but something to fetch all the JSON in one compressed tarball would be nice as well.
item_details + recipe_details + skin_details = ~199MB
in a .tar.bz2 = ~2.3MB
AysonCurrax.3254:
I just got my application properly running and hammering the v1 items api with requests through 4 threads while syncing my dynamic adding of types and flags and covering all 4 currently available languages, and now you do this to me.
Now I feel silly for pouring all the work into making it work as i want it to, and you hand me a way to have it easier. I love you for the much simpler v2 structure nonetheless. It will make my life easier in the long run :P If anything, my knowledge improved while working on it, so i am not even mad.
keep up the good work. glad to see the apis are finally getting some love again.
Gewd.8125:
What is the proper way to handle items that are in commerce but not in items?
AysonCurrax.3254:
What is the proper way to handle items that are in commerce but not in items?
assume fetal position until they are. that is really the best bet i suppose.
StevenL.3761:
Does the listing have buy offers but no sell offers? Discard that listing. It’s not supposed to be listed.
Pat Cavit.9234:
Yeah, sorry about that. We’re working on a plan to clean up the Commerce listings database but obviously we’re being very careful abut it because, y’know, Commerce DB.
StevenL.3761:
Hey, is anyone else running into HTTP “speed bumps”? If I send too many requests, I start seeing HTTP 502 (Gateway Timeout) and I have to wait a minute or two before I can send new requests. Just wondering if this is a technical limitation, or if we’re being throttled.
edit
Must have been a network congestion thing. It doesn’t happen now, even though I tweaked my code to send even more requests. My best record is a full download of the items database for all languages in just under 32 seconds. I’m impressed by how well the API responds to that many concurrent requests.
Pat Cavit.9234:
Hey, is anyone else running into HTTP “speed bumps”? If I send too many requests, I start seeing HTTP 502 (Gateway Timeout) and I have to wait a minute or two before I can send new requests. Just wondering if this is a technical limitation, or if we’re being throttled.
edit
Must have been a network congestion thing. It doesn’t happen now, even though I tweaked my code to send even more requests. My best record is a full download of the items database for all languages in just under 32 seconds. I’m impressed by how well the API responds to that many concurrent requests.
:)
StevenL.3761:
xD sorry. What would be a more democratic delay between each request?
Pat Cavit.9234:
xD sorry. What would be a more democratic delay between each request?
No no, I’m thrilled that you can grab it that fast.
AysonCurrax.3254:
xD sorry. What would be a more democratic delay between each request?
No no, I’m thrilled that you can grab it that fast.
they climbin’ in yo apis,
snatching your items up,
hide your flags,
hide your types,
and hide your game_types too because they are grabbing everybody out here.
Elrey.5472:
I hope too many requests is not what it’s causing the game to lag for everyone else…
AysonCurrax.3254:
I hope too many requests is not what it’s causing the game to lag for everyone else…
I dont think so. atleast I assume that the api is running on a server of its own. Atleast the API is shut down for a little while when the weekly WvW reset happens, so i would think the API Server and the actual Game Server are separate.
Pat Cavit.9234:
API talks to some of the same back end servers, but the public API frontend you hit is totally separate from game systems.
DarkSpirit.7046:
Are there plans to add a “Search” functionality to the items or commerce api?
Getting the item record only through its id is too limiting as items in the game gets added faster than we can update our apps.
DarkSpirit.7046:
Hey folks, we’ve just enabled /v2/items. Documentation isn’t quite ready yet but I’ll be talking to poke and we’ll go from there.
Querying it is identical to the other /v2 APIs.
https://api.guildwars2.com/v2/items/12138
https://api.guildwars2.com/v2/items?page=0
https://api.guildwars2.com/v2/items?page=1&page_size=200
https://api.guildwars2.com/v2/items?ids=12138,57
By the way, your page_size parameter seems to be ignored. X-Page-Size always return 50 when specifying pages.
Dr Ishmael.9685:
Are there plans to add a “Search” functionality to the items or commerce api?
Getting the item record only through its id is too limiting as items in the game gets added faster than we can update our apps.
My database program takes 50 minutes to scrape the entire items API. And that’s on the slow end – Smiley’s takes less than 30 minutes (IIRC).
AND that’s only necessary for the first run after a new build – after that, you just need to diff the item ID list from /v2/items against the IDs in your database and grab the new items (changes to existing items can only occur with a new build).
DarkSpirit.7046:
Are there plans to add a “Search” functionality to the items or commerce api?
Getting the item record only through its id is too limiting as items in the game gets added faster than we can update our apps.
My database program takes 50 minutes to scrape the entire items API. And that’s on the slow end – Smiley’s takes less than 30 minutes (IIRC).
AND that’s only necessary for the first run after a new build – after that, you just need to diff the item ID list from /v2/items against the IDs in your database and grab the new items (changes to existing items can only occur with a new build).
That is not applicable for all situations unless you are volunteering to host your database on the internet for all our mobile/pc applications to access plus promise to keep it up to date as frequently as possible for free. 
I am only kidding of course, third party data providers do exist (e.g. gw2spidy). But in my opinion, relying on third party providers api is not a good long term solution. Just look at how outdated gw2spidy database is right now.
50 to 30 minutes is still too long for a user to wait while the app refreshes its local database, not mentioning mobile apps which would probably take longer and incur a ton of network traffic should the user specify a search for something that is not in its local database and the app decides to refresh it.
Pat Cavit.9234:
Hey folks, we’ve just enabled /v2/items. Documentation isn’t quite ready yet but I’ll be talking to poke and we’ll go from there.
Querying it is identical to the other /v2 APIs.
https://api.guildwars2.com/v2/items/12138
https://api.guildwars2.com/v2/items?page=0
https://api.guildwars2.com/v2/items?page=1&page_size=200
https://api.guildwars2.com/v2/items?ids=12138,57By the way, your page_size parameter seems to be ignored. X-Page-Size always return 50 when specifying pages.
RedBot and my browser agree that it’s fine.
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Content-Language: en
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-Content-Type-Options: nosniff
Link: </v2/items?page=0&page_size=200>; rel=previous,
</v2/items?page=2&page_size=200>; rel=next,
</v2/items?page=1&page_size=200>; rel=self,
</v2/items?page=0&page_size=200>; rel=first,
</v2/items?page=195&page_size=200>; rel=last
X-Page-Total: 196
X-Page-Size: 200
X-Result-Total: 39108
X-Result-Count: 200
Access-Control-Allow-Origin: *
Date: Fri, 24 Oct 2014 18:15:06 GMT
Content-Length: 8706
Looks right to me.
Remfin.4892:
50 to 30 minutes is still too long for a user to wait while the app refreshes its local database, not mentioning mobile apps which would probably take longer and incur a ton of network traffic should the user specify a search for something that is not in its local database and the app decides to refresh it.
It should not take remotely 30-50 minutes on a PC, and it’s not that much data. My program pulls it down in 5-20 seconds (for one language, depending on how my connection is at the time), and the JSON is only 22meg (it’s a lot smaller when gzipped, although I can’t measure that currently).
Mobile may be a different story, but you just have to be smarter about it. But considering how (in)frequent builds are, those numbers aren’t out of the realm of possibility for mobile either.
Dr Ishmael.9685:
50 to 30 minutes is still too long for a user to wait while the app refreshes its local database, not mentioning mobile apps which would probably take longer and incur a ton of network traffic should the user specify a search for something that is not in its local database and the app decides to refresh it.
It should not take remotely 30-50 minutes on a PC, and it’s not that much data. My program pulls it down in 5-20 seconds (for one language, depending on how my connection is at the time), and the JSON is only 22meg (it’s a lot smaller when gzipped, although I can’t measure that currently).
Mobile may be a different story, but you just have to be smarter about it. But considering how (in)frequent builds are, those numbers aren’t out of the realm of possibility for mobile either.
Pulling the data can be done in seconds, yes. But parsing the JSON into a native data structure, computing an MD5 and comparing it to the known MD5 for change detection, running some transformations on the data (e.g. deriving the prefix name from the infix attributes), and posting it to a database (with proper logging and change tracking) takes significantly more time.
My program would certainly run faster if I had a better computer or made some more optimizations to my code – currently I’m building the data structure and computing the MD5 in the same function; if I split the MD5 into its own function, then 90% or more of the items wouldn’t have to be converted into a structure at all.
DarkSpirit.7046:
Hey folks, we’ve just enabled /v2/items. Documentation isn’t quite ready yet but I’ll be talking to poke and we’ll go from there.
Querying it is identical to the other /v2 APIs.
https://api.guildwars2.com/v2/items/12138
https://api.guildwars2.com/v2/items?page=0
https://api.guildwars2.com/v2/items?page=1&page_size=200
https://api.guildwars2.com/v2/items?ids=12138,57By the way, your page_size parameter seems to be ignored. X-Page-Size always return 50 when specifying pages.
RedBot and my browser agree that it’s fine.
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Content-Language: en
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-Content-Type-Options: nosniff
Link: </v2/items?page=0&page_size=200>; rel=previous,
</v2/items?page=2&page_size=200>; rel=next,
</v2/items?page=1&page_size=200>; rel=self,
</v2/items?page=0&page_size=200>; rel=first,
</v2/items?page=195&page_size=200>; rel=last
X-Page-Total: 196
X-Page-Size: 200
X-Result-Total: 39108
X-Result-Count: 200
Access-Control-Allow-Origin: *
Date: Fri, 24 Oct 2014 18:15:06 GMT
Content-Length: 8706Looks right to me.
Really? Perhaps I misunderstood the syntax. Can someone else verify this?
URL: https://api.guildwars2.com/v2/items?page=2&page_size=300
Fiddler:
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Language: en
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-Content-Type-Options: nosniff
Link: </v2/items?page=1&page_size=50>; rel=previous, </v2/items?page=3&page_size=50>; rel=next, </v2/items?page=2&page_size=50>; rel=self, </v2/items?page=0&page_size=50>; rel=first, </v2/items?page=782&page_size=50>; rel=last
X-Page-Total: 783
X-Page-Size: 50
X-Result-Total: 39111
X-Result-Count: 50
Access-Control-Allow-Origin: *
Date: Sat, 25 Oct 2014 01:34:54 GMT
Content-Length: 27415
DarkSpirit.7046:
50 to 30 minutes is still too long for a user to wait while the app refreshes its local database, not mentioning mobile apps which would probably take longer and incur a ton of network traffic should the user specify a search for something that is not in its local database and the app decides to refresh it.
It should not take remotely 30-50 minutes on a PC, and it’s not that much data. My program pulls it down in 5-20 seconds (for one language, depending on how my connection is at the time), and the JSON is only 22meg (it’s a lot smaller when gzipped, although I can’t measure that currently).
Mobile may be a different story, but you just have to be smarter about it. But considering how (in)frequent builds are, those numbers aren’t out of the realm of possibility for mobile either.
Well unfortunately, I am thinking of a mobile app and I don’t think it is fair to expect developers to update their apps immediately after a new build.
In other words, you can expect weeks or months before the app is updated meaning that the new Halloween items would not be searchable until Halloween is over, if the search has to rely on a static local database within the device. Pulling in a huge amount of data through the cellular network and parsing it would be detrimental to the data plans of many users, not to mention how slow it is going to be and all this while displaying a busy cursor prompting the user to wait. That is just bad design coupled with terrible user experience.
Remfin.4892:
300 is an invalid page size; it’s probably substituting in the default.
I’m not sure why you would MD5 item data since AFAIK build+id+lang represents a unique piece of unchanging data; just replace it.
There are other paths to take on mobile, such as a server compiling daily snapshots of the DB to download compressed, detecting when on wifi and updating then, etc.
I think the real problem on mobile would be the risk of ANet breaking your app because your object model couldn’t handle some new item.
DarkSpirit.7046:
300 is an invalid page size; it’s probably substituting in the default.
I’m not sure why you would MD5 item data since AFAIK build+id+lang represents a unique piece of unchanging data; just replace it.
There are other paths to take on mobile, such as a server compiling daily snapshots of the DB to download compressed, detecting when on wifi and updating then, etc.
I think the real problem on mobile would be the risk of ANet breaking your app because your object model couldn’t handle some new item.
I see the same results for 200.
Then you incur secondary cost just to release a free app. You can pass the cost to your users instead, but that is not a good long term plan either. ArenaNet already has such a server, it doesn’t make sense for developers to have to host their own just to support their mobile apps.
Remfin.4892:
If you’re following the link from this forum, it looks like their exit page escapes ampersands in URLs no matter what you do. Paste the URL directly into your browser and you’ll see the 300 page size blows up: "{"text":“page_size out of range. Use page_size values 1 – 200.”}"
As to the rest…I’m simply saying there are ways to work around whatever challenges. And you hardly need to pay money to serve files on the web from a known location (and I don’t mean file sharing services). Even with a paid option (going over the free limits) you’re talking at most dollars a month for such a thing.
smiley.1438:
And that’s on the slow end – Smiley’s takes less than 30 minutes (IIRC).
It does, in fact. My dl is 384 kbit/s btw.
Pat Cavit.9234:
If you’re following the link from this forum, it looks like their exit page escapes ampersands in URLs no matter what you do. Paste the URL directly into your browser and you’ll see the 300 page size blows up: "{"text":“page_size out of range. Use page_size values 1 – 200.”}"
Yup, forums has a link encoding bug and is breaking multiple query params. I’ve verified w/ every link posted so far (after fixing) that all the headers are correct.
DarkSpirit.7046:
If you’re following the link from this forum, it looks like their exit page escapes ampersands in URLs no matter what you do. Paste the URL directly into your browser and you’ll see the 300 page size blows up: "{"text":“page_size out of range. Use page_size values 1 – 200.”}"
Yup, forums has a link encoding bug and is breaking multiple query params. I’ve verified w/ every link posted so far (after fixing) that all the headers are correct.
Yes that must have been the issue. I apologize for the scare.
DarkSpirit.7046:
As to the rest…I’m simply saying there are ways to work around whatever challenges. And you hardly need to pay money to serve files on the web from a known location (and I don’t mean file sharing services). Even with a paid option (going over the free limits) you’re talking at most dollars a month for such a thing.
Yes yes, I know, if you try hard enough and you are not that poor, you can write a mobile app for it, setup your own database web api service, religiously syncs up your database upon new items, and pay the monthly bills, etc., if ArenaNet decides not to provide you with a search functionality.
I hate it, but If that is the expected cost of developing mobile apps with a search functionality then so be it…
Remfin.4892:
Azure WebJob to download database, generate deltas: free
Azure Storage: pennies
1MB/day per mobile client in updates (probably a higher number than reality), free tiers proxying around your Azure storage (avoid Azure bandwidth charges):
AppHarbor: 3000 clients
Heroku: 60000 clients
Offline, in-app searchable database that is never more than two hours out of date for 30MB/transfer a month for the mobile user.
The hard part is about building the app, not paying money to host it.
DarkSpirit.7046:
Azure WebJob to download database, generate deltas: free
Azure Storage: pennies
1MB/day per mobile client in updates (probably a higher number than reality), free tiers proxying around your Azure storage (avoid Azure bandwidth charges):
AppHarbor: 3000 clients
Heroku: 60000 clientsOffline, in-app searchable database that is never more than two hours out of date for 30MB/transfer a month for the mobile user.
The hard part is about building the app, not paying money to host it.
Alright if you say that it is very easy and cheap, so are you volunteering to host it and share out the searchable data to the community?
You can start out with gw2spidy open source, since gw2spidy api provides search functionality, and update it for the v2 API then release the PHP source so that others can also distribute the content. Drakie (gw2spidy owner) and many others would probably be grateful to you for helping him out to revive gw2spidy and you save ArenaNet that work of incorporating search in their api too.
If you can do all that then I would have no choice but to agree with you that setting and hosting it is easy and cheap. It is always easy and cheap when somebody else is doing the work and paying for it at the same time.
Dr Ishmael.9685:
I’m not sure why you would MD5 item data since AFAIK build+id+lang represents a unique piece of unchanging data; just replace it.
Because on my machine, computing and comparing the MD5 takes far less time than updating the database. Also, I want to track each time that an item changes, I don’t want to treat every item as having been changed after every build. Especially when Anet releases 3 builds in the same day.
smiley.1438:
broken pagination is broken.
https://forum-en.guildwars2.com/forum/support/forum/Thread-loaded-as-Empty/first#post2149426
https://forum-en.guildwars2.com/forum/support/forum/No-posts-in-a-thread/first#post2693455
(thread unreadable)