[API Suggestion] Items, Recipes, and Crafting
rodadams.5963:
Let’s see, I’ve seen the following requests:
Items:
- Have an index to which recipes they are created from and/or used in.
- If available from vendor, price in copper/karma for item.
Recipes:
- Clearly state the Crafting Discipline. In particular, bags are hard to “guess” who makes them.
- Include the ‘set’ Mystic Forge recipes.
- Source of Recipe: Discoverable, or bought?
I’ll add in:
- For food items, spell out what the buffs they give, like weapons do.
- Similar for runes, and any other types that don’t specify them.
xanthic.9478:
I am excited by what I see here, as it would make adding new recipes to my crafting script very easy in the future(instead of having to parse the wiki and create custom tools I can just run an update vs the api).
However there are a few changes(more information) that I would like to see.
https://api.guildwars2.com/v1/recipes.json
- Fine as is.
https://api.guildwars2.com/v1/recipe_details.json?recipe_id=3479 example recipe
- Field indicating if the recipe is a refine, part or item – different xp gained for each. Unless type is supposed to fill that function.
- What craft(s) can use this recipe.
- Recipe source. Discovery, learned automatically, learned from recipe(breadcrumb to recipe IE “35 karma, Scholar Tholin – Krongar Pass(Timberline Falls 50-60)”)
https://api.guildwars2.com/v1/items.json
- Fine as is
https://api.guildwars2.com/v1/item_details.json?item_id=13273 example item
- Link to the icon used by the item. Such as https://dfach8bufmqqv.cloudfront.net/gw2/img/content/e414cb68.png
DarkSpirit.7046:
https://api.guildwars2.com/v1/item_details.json?item_id=13273 example item
- Link to the icon used by the item. Such as https://dfach8bufmqqv.cloudfront.net/gw2/img/content/e414cb68.png
For item details, it would be useful, if the item is sold by a vendor, to get the cost of buying it from a vendor. Besides the price of SELLING it to the vendor, which we can also get from the Trading Post API anyway. Include “Gold Cost”, or “Karma Cost”, “Laurel Cost”, or “Skill point Cost”. This would make such derivations possible, for example:
http://www.gw2spidy.com/recipe/3157
I think gw2spidy is using the data provided by gw2db but now that we have an official source, apps can consider switching to use this provided it has the data that they need.
A great bonus would be supporting Mystic Forge recipes. You can expose mystic forge recipes as belonging to its own unique crafting discipline if you like. Sorry if we are asking for too much. 
LadyRhonwyn.2501:
What I miss in this API is a recipe_id for those items that are also created using a recipe.
If you take your recipe example. One of its ingredients is this:
https://api.guildwars2.com/v1/item_details.json?item_id=12812
Which is a crafting material, but one that requires another recipe to craft. It would be nice if there was a recipe_id in there as well, so you can eventually find the base materials of the final crafted item.
DarkSpirit.7046:
What I miss in this API is a recipe_id for those items that are also created using a recipe.
If you take your recipe example. One of its ingredients is this:
https://api.guildwars2.com/v1/item_details.json?item_id=12812
Which is a crafting material, but one that requires another recipe to craft. It would be nice if there was a recipe_id in there as well, so you can eventually find the base materials of the final crafted item.
That is created by recipe_id 3392.
https://api.guildwars2.com/v1/recipe_details.json?recipe_id=3392
You would have to match the output_item_id from the recipe list.
But that would mean you have to download all recipe_details of each recipe to search for the output_item_id of the item that you need to craft. Since you can only request the recipe_details of 1 recipe at a time it is going to be a performance hit for both server and client.
I suggest we either:
1. Allow output_item_id as a parameter to recipe_details.json.
or
2. Like LadyRhonwyn suggests, have a recipe_id field in the item record. That would also mean having a unique value for this field in items that are not craftable.
or
3. Have a way for recipe_details.json to return all recipes in 1 call. Leaving it to the client to search for the required output_item_id.
Charismatic Harm.9683:
I would like to see an API that gives me all of the recipe details for all of the recipes in one JSON file. Currently, the procedure for getting all of the recipe details is…..
Step 1: Call “recipes” to get the list of recipe_id’s
Step 2: Loop through each recipe_id
Step 3: Call “recipe_details” to get the details of each recipe
This method takes a large amount of bandwidth as the current recipe_id list is 6870 items long….each requiring an individual call to get its details.
If a new “recipes_details” API was created or “recipe_details” would allow for the request of multiple recipes at once (say 250), then the number of requests would be dramatically reduced.
If the new “recipes_details” was created that contained ALL details for ALL recipes, then you would only need to call it once to get all of the information.
If the current “recipe_details” allowed for multiple requests (250), then you would need 6870 / 250 = 27.48 or 28 calls.
Either way, 1 or 28 calls, is FAR less than 6870 calls and would reduce bandwidth dramatically.
Edit: I haven’t tried using the “items” and “item_details” API’s yet, but I would hazard a guess that they work the same way.
smiley.1438:
For the items with type Consumable/Unlock, e.g. https://api.guildwars2.com/v1/item_details.json?item_id=9653
- recipe_id of the corresponding recipe which would be unlocked: 3182
- item_id of the crafted item, in this case: 12561
further for the recipes, example from above
https://api.guildwars2.com/v1/recipe_details.json?recipe_id=3182
- item_id of the Consumable which is required to unlock (if applies): 9653
and finally for craftable items
https://api.guildwars2.com/v1/item_details.json?item_id=12561
- recipe_id of the corresponding recipe: 3182
Yoone.9461:
Items: add link to icon, like the current Trading Post API, but working without any GW2 account, and for every item.
smiley.1438:
I’ve forgot the chat code - i know we can calculate them on our own, but it would be more reliable to have them from the original source
Yoone.9461:
I’ve forgot the chat code - i know we can calculate them on our own, but it would be more reliable to have them from the original source
I don’t know if this really is a good idea, it would add unnecessary information to the API. Maybe releasing a function in several languages to convert the ID into the chat code would be more useful!
Rawrfaec.6412:
By extension a link to an item icon is unnecessary as long as we get a resource from which we can fetch an icon corresponding to the item’s ID, which is already given. Having a way to get both icon and chat code would be nice but neither has to be in JSON format.
Yoone.9461:
We can’t get every item icon from the ID, even with the Trading Post API. But we can generate every chat code with the ID.
That is correct. If it never has been sold on the TP, there is no image on there yet. As of yet there is no way to get icons for items not on the TP. I would put good money on it coming fairly soon though.
See the post: https://forum-en.guildwars2.com/forum/community/api/Items-icons-and-in-game-code/first#post2086538
Lethargy.3659:
This isn’t a major problem, but I’ve found an item which is on the TP, but not in the API:
“Sealed Package of Snowballs” (item id 24)
Might this be because the item hasn’t dropped since the christmas event, so it’s not considered discovered (That’s a complete guess at how it’s been implemented)?
DarkSpirit.7046:
I would like to see an API that gives me all of the recipe details for all of the recipes in one JSON file. Currently, the procedure for getting all of the recipe details is…..
Step 1: Call “recipes” to get the list of recipe_id’s
Step 2: Loop through each recipe_id
Step 3: Call “recipe_details” to get the details of each recipeThis method takes a large amount of bandwidth as the current recipe_id list is 6870 items long….each requiring an individual call to get its details.
I would like to second this suggestion and also include item_details for this suggestion also.
Considering that we would need to be registered developers with api key and set quota, this suggestion for being able to get all the item details in 1 call and recipe details in 1 call, becomes even more important.
DarkSpirit.7046:
I have posted on another suggestion thread before but since now we have an official suggestion thread for items, here goes:
For item details, it would be useful, if the item is sold by a vendor, to get the cost of buying it from a vendor. Besides the price of SELLING it to the vendor, which we can also get from the Trading Post API anyway. Include “Gold Cost”, or “Karma Cost”, “Laurel Cost”, or “Skill point Cost”. This would make such derivations possible, for example:
http://www.gw2spidy.com/recipe/3157
I think gw2spidy is using the data provided by gw2db but now that we have an official source, apps can consider switching to use this provided it has the data that they need.
Another great feature would be supporting Mystic Forge recipes. You can expose mystic forge recipes as belonging to its own unique crafting discipline if you like. Sorry if we are asking for too much.
Dr Ishmael.9685:
The problem with including vendor costs is that different vendors can sell the same item at different prices. I’m not talking just coin versus karma, but I know at one time during beta there was an item that different vendors sold for different amounts of coin.
A better way to present that data would be via a vendor_details.json API, that would key off of actor_id or something and list all the items the vendor sells, along with the prices.
smiley.1438:
I know at one time during beta there was an item that different vendors sold for different amounts of coin.
Golem-In-A-Box is such an item – sold in Metrica Province at 2 different vendors for different price …and it’s missing in the Items-API :o
DarkSpirit.7046:
The problem with including vendor costs is that different vendors can sell the same item at different prices. I’m not talking just coin versus karma, but I know at one time during beta there was an item that different vendors sold for different amounts of coin.
A better way to present that data would be via a vendor_details.json API, that would key off of actor_id or something and list all the items the vendor sells, along with the prices.
Alternatively we can have a vendor array on each item record with each element in the array containing the vendor id and the gold/karma/laurel/skill point cost for the item.
This would be similar to an item record with something like this:
{...,"SoldBy":[{"NPCExternalID":700000,"KarmaCost":49},{"NPCExternalID":700001,"KarmaCost":77}]}
{...,"SoldBy":[{"NPCExternalID":700000,"GoldCost":8}]}
The advantage of this approach is that I won’t need to load the vendor.json, when I am not interested in vendor information, in order to generate a tree view like the one shown in http://www.gw2spidy.com/recipe/3157
SLAYER.3941:
Receiving the image / icon for an item is totally necessary
smiley.1438:
Would be nice if Arenanet would release an official asset kit containing all those icons for use with the API or some sort of tool to pull the data from the gw2.dat. People have already started to pull the data in some ways from the AH which i believe wasn’t intended. Official sources would also help the wikis
xanthic.9478:
For these API to be useful for me I would need the additional information in bold. This is what I envision needing to automate my crafting guides recipes.
get all valid/known recipes via recipes.json
via recipe_details.json get each(new) recipes item_id created, skill level, craft required(eg cooking), recipe source(eg discovery, learned automatically, learned from recipe – with source), craft type from refine, part and item(they each give different xp)
Sample fields:
“craft”:“cooking”
Possibly values: cooking, jewelcrafting, artificing, leatherworking, armorcrafting, tailoring, weaponcrafting, huntsman
“source”:“discovery”
Possible values: discovery, learned, recipe name eg “Recipe: Ooze Custard”
“craft_type”:“item”
Possible values: part, refine, item. This may be redundant if “type” already implies that information but it would need possible results documented somewhere.
DarkSpirit.7046:
Would be nice if Arenanet would release an official asset kit containing all those icons for use with the API or some sort of tool to pull the data from the gw2.dat. People have already started to pull the data in some ways from the AH which i believe wasn’t intended. Official sources would also help the wikis
What Zicore’s TP notifier does is to cache those images in a folder whenever his app tries to fetch an item image. This way, the app actually runs faster the more the user uses it at the cost of some disk space (which is cheap nowadays anyway). My image cache folder has 4738 PNG files right now.
Healix.5819:
When caching many small files, you can save a little bit of disk space by creating one large file. Files are allocated a specific amount of disk space, which for mostly everyone is going to be in intervals of 4KB. If you have a 1 byte file for example, it’s going to use up 4 KB.
For ~4700 3-8KB files, you could save anywhere from 5-10 MB of space, assuming 64×64 icons.
In my case, I’m caching 24,853 raw responses that are just a few 100 bytes in length each. In total, it’s only 11 MB, but since they’re all separate files, it’s actually 97 MB. In database form, it’s 4 MB.
DarkSpirit.7046:
When caching many small files, you can save a little bit of disk space by creating one large file. Files are allocated a specific amount of disk space, which for mostly everyone is going to be in intervals of 4KB. If you have a 1 byte file for example, it’s going to use up 4 KB.
For ~4700 3-8KB files, you could save anywhere from 5-10 MB of space, assuming 64×64 icons.
In my case, I’m caching 24,853 raw responses that are just a few 100 bytes in length each. In total, it’s only 11 MB, but since they’re all separate files, it’s actually 97 MB. In database form, it’s 4 MB.
That is cool, but in the case of my app that would be a little in the realm of adding more complexity for little gain since hard disk space is very cheap nowadays anyway. Squeezing better performance is where I have been investing time into.
xanthic.9478:
Tonights changes added everything I was hoping for. Crafting guide should get automated recipe lists tomorrow and then I won’t have to touch it again (except to add any new cooking recipe locations)
DarkSpirit.7046:
I am still holding out hope that they would provide NPC vendor “SoldBy” info for items soon.
I would really like to be able to come up with something like this:
http://www.gw2spidy.com/recipe/3157
…with the data provided.
I already have the code developed for this against the gw2db data but their data is missing some NPC vendor prices for some items so I had to supplement it.
As a bonus, if they can also provide Mystic Forge recipes that would be even better. I tried mixing the MF recipes into the crafting recipes and my code came up with new cheaper ways to make some of the named exotics, which is very cool.
havom.1532:
recipe_details: Suggestion / Missing
For the recipe_details I´m missing: The category of recipe: Insigmia, refinement, Upgrade component, Ring, and so on (such as seen here: http://wiki.guildwars2.com/wiki/File:Jeweler_Crafting_Pane.png). Even mystic forge can be applied there.
Dr Ishmael.9685:
That’s the “type” element in recipe_details.json. Mystic Forge recipes are not included in the API at this time.
DarkSpirit.7046:
I have one more suggestion. Right now there is no way for me to find out the Recipe item itself that is required to unlock a particular recipe.
For example, take the recipe for Candy Corn Custard:
https://api.guildwars2.com/v1/recipe_details.json?recipe_id=6472
…and the Recipe item to unlock the above recipe:
https://api.guildwars2.com/v1/item_details.json?item_id=36103
There is currently no way for me to link one to the other, short of doing fancy regex matches. It would be great if the recipe item can specify the recipe id that it actually unlocks OR the recipe itself, with the “LearnedFromItem” flag, can specify the recipe item that is used to unlock it.
Thanks.
Dr Ishmael.9685:
There is currently no way for me to link one to the other, short of doing fancy regex matches.
Even that wouldn’t work all the time. Example: Recipe: Atzintli’s Spear teaches the recipe for Quetzal Harpoon. Some others aren’t quite as bad (Recipe: Eggs Beetletun teaches Beetletun Omelette), but there are a lot like this.
I second this request.
smiley.1438:
Thats pretty much what i’ve already suggested in post #3, isn’t it?
DarkSpirit.7046:
Thats pretty much what i’ve already suggested in post #3, isn’t it?
That just goes to show how much the community needs this.
EDIT: If we have this, and the “SoldBy” property for items, then apps would be able to start providing useful information on how + where to unlock recipes.
If not the “SoldBy” property, we can use the Vendor API, if that is provided instead.
https://forum-en.guildwars2.com/forum/community/api/API-Suggestion-Vendor-API/first#post2175422
Dr Ishmael.9685:
Unlocks now list the color_id/recipe_id they unlock. Whoo!
DarkSpirit.7046:
That was a fast turnaround! Woot!
The new unlock_type enum:
public enum GW2APIUnlockTypeEnum
{
CraftingRecipe,
Dye,
BagSlot,
BankTab
}
Dr Ishmael.9685:
And now it’s obvious that there are a lot of recipe sheets missing. Both of the ones I linked above (Eggs Beetletun, Atzintli’s Spear) are missing, as is Eda’s Apple Pie Recipe. Some of the recipes are listed (recipe_ids are 2968 for Beetletun Omelette and 2903 for Eda’s Apple Pie) and they have the LearnedFromItem flag, but the items API discovery mechanism still has some big flaws.
DarkSpirit.7046:
Yes but items are getting “discovered” with each day so at least there is hope. The only thing that I need now from the items API is either a “SoldBy” property (with its vendor acquisition cost) for vendors OR a vendor API that lists the items each vendor sells and its cost.
With that I would be able to show where and how to unlock recipes and also be able to generate a tree view for minimum crafting cost for craftable items, like in gw2spidy.
I believe gw2spidy had to supplement their data that they got from gw2db since the SoldBy property in gw2db items API is not very complete.
Dr Ishmael.9685:
I made a spreadsheet of all the LearnedFromItem recipe details and added the item_id of the consumable that unlocks it, as well as the name of the output_item_id.
https://docs.google.com/file/d/0B5NobLNMdY89TnpzVC00Zm1vemc/edit?usp=sharing
Looks like the majority of the recipes with an unlisted Unlock are where the recipe sheet is purchased for karma, although there are still some Bulk and Feast recipes with an unlisted Unlock even though you get those recipe sheets from the Mystic Forge.
Paranaix.8374:
As I missed this thread on the first glance I beg for pardon and simply repost my suggestion here:
It would be kind of nice if you provide more information for bags, especially their type.
Currently a definition looks e.g like this one:
{u’restrictions’: [], u’name’: u’12 Slot Equipment Box’, u’level’: u’0’, u’rarity’: u’Fine’, u’bag’: {u’no_sell_or_sort’: u’0’, u’size’: u’12’}
As you can see there is no information about that equipment go firstly in this bag .
For "Invisible" Bags such information is already provided (u’no_sell_or_sort’: u’0’) why not for other bag types?
With best regards, Paranaix
xanthic.9478:
I’d like to see a bulk get(like the TP api has) so I am not doing 10,000 individual api calls when rebuilding recipe sheets. EG https://github.com/xanthics/gw2craft/blob/master/create_recipes.py
elentras.4965:
I’m bored about items pictures/icons…. I mean, if I want to get the pictures for each object I have to connect to the TradingPost API and this is pretty painfull to do.
I use certificates, https connection and simulate from my application the connection through a browser. But I would be please to simply get them from the items hash, as a link to the picture.
I just want to do it one time, but I can understand than beginners will request it all the time and slow down the assets server. Please, find a solution about it !
I’m thinking about blocking ip if doing two times the same requestor more.
Dr Ishmael.9685:
As far as icons go, I’d be perfectly happy with a simple icon_file_id. I’ve already got GW2DatBrowser to extract the files, so I don’t need the file itself presented through the API or on a cloud server or anything like that.
ozma.3498:
Glad to see there is already a lot of interest for icons to be added to item_details.json – I was just going to make a post suggesting it.
Since ArenaNet are encouraging people to hotlink to their map tiles rather than mirror them, I have hope that they will add icon support to the current items API. As many people have stated, at the minute it’s very much needed.
While I personally have no issues using GW2DatBrowser to get the icons, I don’t think it would be good practice to have icon IDs in the API without also providing those icons directly.
The best implementation I can think of would be to have something like “icon_id” in item_details.json, and create item_icons.json with URLs to ArenaNet-hosted PNG files that correspond with the IDs.
Kawakipik.7908:
Hi
discovering new recipes becomes a technical chalenge.
Yamagawa.5941:
How is discovering we recipes any sort of challenge?
Maintain a list of known recipe IDs. If the list changes over time, you know what was added/removed.
You can spot changed recipes in a similar way… You just need a few thousand API calls.
//Yamagawa
LadyRhonwyn.2501:
I just found out there’s a new field in the item details json… (two in fact..)
“icon_file_id”
“icon_file_signature”
Now a way to figure out how to use them…
Healix.5819:
icon_file_id is the id of the file in the gw2.dat file. You would have to extra the dat file for that number to be of any use to you.
I assume they’ll release the icons in the same way they did map tiles.
LadyRhonwyn.2501:
Apparently I was ahead of the official update post
Now I’ll need to update my entire list of 26429 items to get the icon id and signature…
Nemetona.6234:
See https://forum-en.guildwars2.com/forum/community/api/Render-service-for-item-icons-404 for ANet Icon Render Service URLs.
ozma.3498:
I’d like to see some kind of filtering options for items.json. For example, being able to only show item IDs for items that are exotic by calling items.json?rarity=6 (or something to that effect). Ideally we’d be able to add as many parameters as we want – for example, Exotic gloves that are heavy armor and usable by Sylvari: items.json?rarity=6&weight=3&slot=4&usable_by=3. Just made up numbers, but you get the idea.
I’d also like to see the ability to retrieve more than one item’s data at once using item_details.json?item_id=[3012,7610,3478,1029] or something similar.
Faedrivin.5382:
Maybe I just miss it, but I still can’t find the Mystic Forge recipes in the API.
If I just don’t see them, can anyone tell me where to look for them? If they are still not inside I hereby support the request for Mystic Forge recipes. That would be really good.
Dr Ishmael.9685:
Mystic Forge recipes are not in the API. Only crafting.
DarkSpirit.7046:
Unfortunately many of the mystic forge recipes are still shrouded in mystery and even the current information on the wiki is not entirely correct.
I may be wrong, but I assume ArenaNet would not release those information as they prefer players to experiment on the mystic forge and share info.
smiley.1438:
Please bring mystic forge recipes to the recipes API! Please! It gets annoying :o
nino.3946:
You would have to match the output_item_id from the recipe list.
But that would mean you have to download all recipe_details of each recipe to search for the output_item_id of the item that you need to craft. Since you can only request the recipe_details of 1 recipe at a time it is going to be a performance hit for both server and client.
It would only be a performance hit if you made an API request each time . Guild Wars 2 doesn’t release new items that, so you could easily create a cache, or alternatively create a database and sync it with theirs (using the API). This would even eliminate the need for future requests.
I suggest we either:
1. Allow output_item_id as a parameter to recipe_details.json.
Yes please.
2. Like LadyRhonwyn suggests, have a recipe_id field in the item record. That would also mean having a unique value for this field in items that are not craftable.
This could be a bit tricky, as we don’t know how the implementation details of the way items are handled, it’s very variable, but it could cause stress on the server.
3. Have a way for recipe_details.json to return all recipes in 1 call. Leaving it to the client to search for the required output_item_id.
This would create a similar problem that you’re trying to fix:
Even though the amount of data per recipe is quite minuscule (I’ll estimate it’s between 200 and 300 bytes), imagine multiplying that by 7620 (at the time this post was written), this easily blows up to 1,905,000 bytes (~1.8 Mb), which is a much much bigger load.
However, the very size is not the only problem, but the problem is that someone could misuse this feature, to make a whole bunch of requests repeatedly (which adds stress on the transfer layer and the database for GW2 servers).
Of course, you could argue that this type of request could be made once after each update and used for syncing or caching the data (like a similar observation to your first suggestion), but they can’t be certain users would use it that way, which leaves them at risk of increasing load on their servers.
Your last idea would be very good, if ArenaNet decided to implement the practice of API points (or something similar).
Lazarus.9716:
TL;DR: Batch detail downloads, queries for only objects that have changed.
I have a number of suggestions on the technical/feed side that would significantly improve mobile app performance and capabilities. These are largely motivated by the fact that mobile applications really to keep data from the API (items, recipes, etc.) stored locally in a database. This is for a number of reasons.
1. Mobile applications should really be usable without a network connection. That’s pretty much becoming a standard requirement for apps these days.
2. Functionality I’m implementing would prohibitively expensive in terms of network turnaround. Right now as I’m testing it takes almost 20 minutes to download full item detail for the ~30k items in the game. That’s not even including images, or the ~6k recipes to make them. So advanced search capabilities (filtering by rarity for example) would be impossible without the full item set requires all the data if there’s no search API.
3. Hitting your servers for information that rarely (if ever) changes is an unnecessary load on your infrastructure. Reducing the number of requests would simply make the service faster for everyone and save you infrastructure costs. Given the search example, I can do search much faster and efficiently with local data (and with better functionality).
4. Mobile applications should able query for changed data (item details, for example) periodically so that we don’t have to re-seed the local database and release an app update. User’s simply aren’t going to sit around for 20 minutes while the app reloads everything and sees what changed and that’s the only option right now.
So, given all that, here are my suggestions.
1. First off, for v2 of the API I would suggest significant changes to the endpoints, response formats, etc. It would make use of the API much clearer, reduce the number of endpoints, and make applications more robust. Feel free to PM me if you want to talk about that.
2. Any API endpoint that returns a large number of IDs that require another endpoint call for the actual object detail (items for example) really needs to have a variant that returns all of the detail for the items in one response. Having to do 30k item detail endpoint calls must be crushing your servers and certainly not performing well for application users considering it takes 20 minutes to download.
3. Add a changedSince query parameter that specifies returning only ids/objects that have changed since a certain time. Unfortunately there’s no standard URL-friendly format. I’d suggest YYYY-MM-DD in UTC. If your database doesn’t keep track of when an item is updated, you might want to try to convince your DBA’s to add one. It’s a pretty standard content management feature.
4. Add an endpoint that returns an array of all the IDs/signatures that are valid for the render service. Add a changedSince parameter as well. If that’s not feasible, add it to the render endpoint call and return an empty document to avoid the ~6k download and the need to compare it on the client side to the stored asset.
Sorry for the wall of text. I think that’s about it for now. These features would make mobile developers’ lives much easier and a much better experience for end-users.
Keep up the good work.
smiley.1438:
Right now as I’m testing it takes almost 20 minutes to download full item detail for the ~30k items in the game.
Wait WHAT? Only 20 minutes? A full update takes us about 20 hours (for all 4 languages of course at ~2-3 seconds per request, wiki check and recipe update not included)
I still think, the most important feature would be an API which provides a list of the changed items/events/recipes since the last patch, so that we can safely do incremental rather than full updates after updates where we don’t exactly know what has changed.
Lazarus.9716:
Yeah, the items call returns in about 2-3 seconds, and the ~30k details requests process at ~24/second. I’m only doing English right now as I just created the data model, and also not doing recipe or other ancillary requests. But it sounds like your performance is way off mine. It’s possible that they’re throttling based on requesting IP address or domain if you’ve hit their service a lot or for a while.
But, yeah, change information would be good for us and for them. The biggest thing, though, is the N+1 endpoint calls. My rough estimate is that a single response would only be about 18MB, could easily be cached on their side and would download in seconds rather than minutes or hours in your case.
The 20 minutes, btw is on ethernet on my dev machine. I would hate to even think how long it would take on LTE or even 3G mobile service. (Remember, we yanks live in the mobile stone age compared to Europe.)
Stefan Larimore.6872:
Hi Lazarus, good suggestions.
I am curious about your ideas for #1. At the time we roll out the OAuth-based APIs or soon after there will probably be an opportunity to also revisit the existing un-auth APIs. Feel free to PM me your ideas or post on the forum itself.
Regarding your idea #2, we have talked about doing this at some point. We wanted to get the fully factored information out there first, and then consider ways to fetch things in bulk later.
For your #3 changedSince, tracking new items or recipes etc is easy – but probably of marginal value since you guys can already do that yourself. Implementing a changedSince for existing items is actually hard given where the API sits relative to the other components. It is something I will think about though.
Thanks!
DarkSpirit.7046:
Here is what I did for my trading post app, I downloaded the entire items and recipes databases into JSON files then I deserialize them async upon app startup. Chances are, the items would not change but more items could be added at a later date. If I find an item in the TP that is not in my items database, the code query the API server on the Internet. If the item is found from the API server, then my database is updated automatically.
This way, the app “learns” and updates its database through usage. And of course, I also try to release new versions of the app with updated items/recipes databases from time to time.
Dr Ishmael.9685:
The obvious gap in a system like DarkSpirit’s is, of course, when an already known item/recipe changes. This happened to a lot of them on Sept. 3, when all existing equipment with Magic Find had their attributes removed, upgrade components that affected MF had their attributes/effects modified, and the recipes that crafted MF equipment were updated with a new output_item_id.
This is the gap that most of us (I assume) are covering by re-downloading the full item/recipe APIs after every release, just to be certain we catch all possible changes. Any system that could tell us which items had changed would make this step much more efficient. One option that I remember being mentioned a while back is to add an MD5 hash of the item/recipe’s JSON data to the API (maybe in a new endpoint like items_md5.json/recipes_md5.json). That way, we can easily tell which items/recipes changed by comparing their MD5 hash between releases.
Drakma.1549:
That is exactly what I do. I download the json and get an md5 hash. I then compare to my database to see if the hash changed. If it has than I update.
Only one problem tho. I still need to download every item detail.
Adding an md5 hash to the items.json would be a huge help.
DarkSpirit.7046:
The obvious gap in a system like DarkSpirit’s is, of course, when an already known item/recipe changes. This happened to a lot of them on Sept. 3, when all existing equipment with Magic Find had their attributes removed, upgrade components that affected MF had their attributes/effects modified, and the recipes that crafted MF equipment were updated with a new output_item_id.
I would call that an exception rather than the norm. It is rare for them to change so many items at one go. Having said that, it is probably wise to do a full item/recipe update after a massive item patch release like that one.
If you operate under the assumption that the item/recipe data themselves rarely change, then that should not be a big issue.
DarkSpirit.7046:
That is exactly what I do. I download the json and get an md5 hash. I then compare to my database to see if the hash changed. If it has than I update.
Only one problem tho. I still need to download every item detail.
Adding an md5 hash to the items.json would be a huge help.
If you still need to download every item detail, then there is no advantage in calculating the md5 hash is there? Afterall, the performance bottleneck is probably the internet access to download every item.
smiley.1438:
The obvious gap in a system like DarkSpirit’s is, of course, when an already known item/recipe changes. This happened to a lot of them on Sept. 3, when all existing equipment with Magic Find had their attributes removed, upgrade components that affected MF had their attributes/effects modified, and the recipes that crafted MF equipment were updated with a new output_item_id.
I would call that an exception rather than the norm. It is rare for them to change so many items at one go. Having said that, it is probably wise to do a full item/recipe update after a massive item patch release like that one.
If you operate under the assumption that the item/recipe data themselves rarely change, then that should not be a big issue.
You guys seem not to know about translation issues and stuff ^.^
At least the german translations and possibly also the french and spanish change with nearly every patch and so they give us a hard time in the wikis to find and move all that stuff. So once more the suggestion: An API which delivers all IDs of changed items for a given build (and therefore an API which delivers the recent build numbers – this could be done by the already existing build.json)
Dr Ishmael.9685:
I am aware of the translation issues, Smiley. I just used the MF change as an example because it was recent and still on everyone’s minds.
smiley.1438:
I belive at least you are aware, Ish. My comment pointed more at this:
I would call that an exception rather than the norm. It is rare for them to change so many items at one go.
DarkSpirit.7046:
I belive at least you are aware, Ish.
I would call that an exception rather than the norm. It is rare for them to change so many items at one go.
I am also aware of the other languages. How often do multiple items change? I am not referring to items being added as they are discovered by players, I am referring to the existing items themselves being changed.
Other than that one-time MF patch, do the items themselves change often? If they do, then there isn’t much of a point to optimize performance by saving all items/recipes as json files in the first place.
smiley.1438:
There have been countless renamings of several items in the german translation (not only) recently. Some items have been renamed more than once. So if you’d like to maintain a database where you’re able to search items by name, a full update after patches seems to be mandatory.
DarkSpirit.7046:
If the items are renamed from the api, then I presume they were cascading down from the renaming of the items in the game code. Assuming the patch note is detail enough, then we should expect a list of items that are impacted in the German client, unfortunately they are not. I can certainly understand why people are asking for this.
DarkSpirit.7046:
Suggestion: Indicate time-gated recipes, for example the recipe to create Lump of Mithrillium.
smiley.1438:
Suggestion: Indicate time-gated recipes, for example the recipe to create Lump of Mithrillium.
+1
The “flags” Array is under-used here anyway
HackworthGW.8251:
Right now as I’m testing it takes almost 20 minutes to download full item detail for the ~30k items in the game.
Wait WHAT? Only 20 minutes? A full update takes us about 20 hours (for all 4 languages of course at ~2-3 seconds per request, wiki check and recipe update not included)
I still think, the most important feature would be an API which provides a list of the changed items/events/recipes since the last patch, so that we can safely do incremental rather than full updates after updates where we don’t exactly know what has changed.
I’ve been starting a tool of my own (in .net), and right now I’m getting ~300 item_details/minute with a naive implementation, synchronous from 1 machine, using 5-6 kB/s down, 1 kB/s up. That’s just under 100 minutes for the full item set per language, or 6.5 hours for 4 languages. Unless flood control sets in at some point, getting the item_details should not be very time-consuming with let’s say 10 parallel tasks, ~40 minutes.
By the way, “1 big download with changes since the last patch” would be useful, but not to everyone. Let’s say someone misses the first patch for example, a stand-alone app that updates manually, or if 2 patches with item changes happened very close to each other (same day or so); or if you are just starting out populating your local database; or you simply want a fresh local database for whatever reason. In those cases, a big download with simply the current state of all items would be best.
smiley.1438:
I made this more clear in this post: https://forum-en.guildwars2.com/forum/community/api/API-Suggestion-Items-Recipes-and-Crafting/2863618
tl;dr: an API which delivers the IDs of changed items starting from a given build number. build.json could be extended and used for this.
Bandlero.6312:
Right now as I’m testing it takes almost 20 minutes to download full item detail for the ~30k items in the game.
Wait WHAT? Only 20 minutes? A full update takes us about 20 hours (for all 4 languages of course at ~2-3 seconds per request, wiki check and recipe update not included)
I still think, the most important feature would be an API which provides a list of the changed items/events/recipes since the last patch, so that we can safely do incremental rather than full updates after updates where we don’t exactly know what has changed.I’ve been starting a tool of my own (in .net), and right now I’m getting ~300 item_details/minute with a naive implementation, synchronous from 1 machine, using 5-6 kB/s down, 1 kB/s up. That’s just under 100 minutes for the full item set per language, or 6.5 hours for 4 languages. Unless flood control sets in at some point, getting the item_details should not be very time-consuming with let’s say 10 parallel tasks, ~40 minutes.
By the way, “1 big download with changes since the last patch” would be useful, but not to everyone. Let’s say someone misses the first patch for example, a stand-alone app that updates manually, or if 2 patches with item changes happened very close to each other (same day or so); or if you are just starting out populating your local database; or you simply want a fresh local database for whatever reason. In those cases, a big download with simply the current state of all items would be best.
I wrote some code out in .Net to test this on my own against the recipe API. Using threading (with multiple web requests each on their own thread; concurrently), I was able to download the entire ~7.7k recipes + recipe details + output item details + ingredient details (about 30,800 requests to the API) in about 30 minutes. It was just some code I quickly hammered out, but I could probably clean it to shave it down to 20 minutes.
Of course, there’s always the bottleneck of the internet itself, and I do wonder at what point Anet’s servers will perceive a massive amount of near-instantaneous traffic as a DDOS attack. I would assume that over 1,000 API requests per minute isn’t appreciated by them anyhow, and it’s not code I’d be willing to use again (even infrequently.)
Honestly, in the current implementations, the recipe/item APIs are bad imo. There’s no real single item search outside of knowing the recipe/item ids (i.e. no request by name (like the guild API supports), by crafting profession, etc.) Inefficient and lack of serving data in a meaningful and useful manner. This is why I’ve yet to release my Recipe/Item related tools to my application’s users. I won’t force a large download on them to their computers (containing the entire recipe/item data that might need to be frequently re-downloaded/updated), and neither do I wish to host a synced database on my own servers to do what the API is supposed to be doing.
Really, it’s poor design (be it the over 7.7k recipes in the recipe list or 30k+ items in the item list) to serve just a list of ids that then necessitate making individual requests for each individual item’s/recipe’s details (that then each might also necessitate requesting multiple sub-items.)
Dr Ishmael.9685:
There’s no real single item search outside of knowing the recipe/item ids (i.e. no request by name (like the guild API supports), by crafting profession, etc.)
Requesting by name wouldn’t make sense for items, because item names are not unique and you could get any number of items returned. Guild names must be unique, so you know requesting a guild name will return exactly 1 guild (or 0 if it doesn’t exist).
Bandlero.6312:
There’s no real single item search outside of knowing the recipe/item ids (i.e. no request by name (like the guild API supports), by crafting profession, etc.)
Requesting by name wouldn’t make sense for items, because item names are not unique and you could get any number of items returned. Guild names must be unique, so you know requesting a guild name will return exactly 1 guild (or 0 if it doesn’t exist).
Yea, for single items and names to work each item would have to have a unique name.
What I meant by requesting by name is that you could pass a parameter (for example) like Rabid, and have the API return to you a subset of the items containing Rabid in the name. Instead of serving the entire list of items and recursively going through items.json and item_details.json just to locate items with Rabid.
We definitely need more support for additional optional parameters to pass to the API.
Dr Ishmael.9685:
I assume your actual intent with that is the ability to query all items with Rabid attributes? You’re still kitten a bit in that, due to named exotics and karma items that don’t include the stat prefix in their names.
Khisanth.2948:
I wrote some code out in .Net to test this on my own against the recipe API. Using threading (with multiple web requests each on their own thread; concurrently), I was able to download the entire ~7.7k recipes + recipe details + output item details + ingredient details (about 30,800 requests to the API) in about 30 minutes. It was just some code I quickly hammered out, but I could probably clean it to shave it down to 20 minutes.
On what kind of connection?
If I use my home connection the full recipes + output items + ingredients takes around 40 minutes but on my web host it takes around 18m. Exact same code.
Looking at the data I have it should only be 15,738 requests. Where is your extra 15k coming from?
Bandlero.6312:
I wrote some code out in .Net to test this on my own against the recipe API. Using threading (with multiple web requests each on their own thread; concurrently), I was able to download the entire ~7.7k recipes + recipe details + output item details + ingredient details (about 30,800 requests to the API) in about 30 minutes. It was just some code I quickly hammered out, but I could probably clean it to shave it down to 20 minutes.
On what kind of connection?
If I use my home connection the full recipes + output items + ingredients takes around 40 minutes but on my web host it takes around 18m. Exact same code.
Looking at the data I have it should only be 15,738 requests. Where is your extra 15k coming from?
For my testing, I used the recipes API (grabbing that file is 1 request), then grabbed the details for each recipe (about 7.7k requests), grabbed the output item details (another ~7.7k requests), then grabbed the ingredient details for each recipe – each recipe has on average 3 ingredients for an additional ~15k to 23k requests. This test did not take into account caching already “known” data and was just a brute force test of the speed/time to download data from this API (i.e. like if this was a first time downloading from the API).
My connection is a dedicated business DSL with a 45 Mbps down / 10 Mbps up.
And to the person (sorry I forgot the name as I write this) that replied to my thought about searching items by name – yea you would still potentially receive a large amount of items to have to iterate through; however, it would still be a much smaller subset of data, more quicker, and more efficient than the current method of having to search through all 7.7k recipes (and the outputs, etc.) or 30k+ items to locate each item with “rabid.” Even 200 items with “rabid” is less than having to iterate through the entire recipes and its structure or the items.
If the APIs functioned a lot more like the Trading Post does in game, that would be the improvement. We should have URL parameters on the APIs that allow for things such as, “?item_name=rabid”, “?item_level=80”, “?item_rarity=legendary”, “?item_type=armor”, etc.; that allows us to retrieve only a subset of the data instead of having to iterate through the entire collection. Honestly, I’m assuming there’s an SQL database driving the back-end of the API, so handling these sorts of requests would minimal additional load on the servers. SELECT * FROM [Items] WHERE [Name] LIKE ‘rabid’.
I don’t know, perhaps my degree in ISE/DBA is making me over-think this, but a system like this is exactly what I’m trained/educated to analyze and fix.
Chiuna.2538:
I would like to suggest/request that when looking up armor pieces, the item details API shows the number of dyes that can be applied to them.
DasPatAttack.2516:
hi guys,
thanks for sharing your API knowledge.
I was wondering how the karma cost and vendor cost can be found? I manage to get all the information using the official APIs and spidy APIs, but no way I can get these remaining informations.
Any idea would be appreciated.
smiley.1438:
I was wondering how the karma cost and vendor cost can be found? I manage to get all the information using the official APIs and spidy APIs, but no way I can get these remaining informations.
Any idea would be appreciated.
There’s currently no official (or legal) way to get the price of an item. I think this has been requested more than once over here. → see first post
Mehlox.1756:
I made this more clear in this post: https://forum-en.guildwars2.com/forum/community/api/API-Suggestion-Items-Recipes-and-Crafting/2863618
tl;dr: an API which delivers the IDs of changed items starting from a given build number. build.json could be extended and used for this.
Push – Great idea. We really need this. “Changelog” for Skills would be great too.
Leylin.3901:
I hope I am in the correct place to add a suggestion for the skin_details.
It would be great to have an attribute skinset or something like that.
This way you could group the skins by their set (for example: all embroidered items). Currently I am missing this option in the game as well as in the API.
Until this is implemented (if it will be) there is no other way than to take all but the last word of the skin as name of the skin set.
Has anyone experienced if this approach is more or less reliable?
Has anyone another idea to get the name of the skin sets?
Sariel V.7024:
I hope I am in the correct place to add a suggestion for the skin_details.
It would be great to have an attribute skinset or something like that.
This way you could group the skins by their set (for example: all embroidered items). Currently I am missing this option in the game as well as in the API.
Until this is implemented (if it will be) there is no other way than to take all but the last word of the skin as name of the skin set.
Has anyone experienced if this approach is more or less reliable?
Has anyone another idea to get the name of the skin sets?
I’ve been digging into skins for months now, and I’ve come to one conclusion regarding sets: sets are somewhat arbitrary.
1. Most skins do conform to having one obvious set name. However, most skins were also part of at least two sets – one for PvE, and one for PvP. Membership in both sets was not always guaranteed. For example, the Bandit weapons overlap with Makeshift weapons, but Makeshift Spear has no PvE counterpart and remains named Makeshift instead of Bandit.
2. Flame weapons did not include SteamFire until SteamFire was renamed a few months ago, even though they shared obvious visual similarities.
3. The Phantasmal set on the wiki only includes weapons that appear on Mesmer phantasms, exclude several that share the same obvious purple smoke characteristic, and NONE HAVE PHANTASMAL IN THEIR NAME.
There’s plenty more like this. So you can do what I do and just go with what seems obvious, because none of these sets are officially established outside of the collections and cultural merchants.
Flokkie.9675:
All the resent changes… I welcome them, most r very useful and if not they don’t bother me.
But with the introduction of the endless tonics also came a shortage in space. If you want to collect (all) tonics it could start costing you space. So my solution: create, like has been done with the mini’s and finishers, a separate tab were you can store all you endless tonics and when you want to use one, you take it out ( a copy) so it becomes like a collectable.
Don’t really know if this is the right place for this suggestion but….
Sariel V.7024:
Some further followup on skinsets: there are several examples of skins using similar or identical models to skins in existing sets that don’t match the naming conventions:
Glyphic: X6-31 Beta (Glyphic Speargun with differing partical effects and color)
Verdant: Rockweed Spire (Verdant Trident with leaves on the haft and different color)
Flame: Flamebelcher (Flame Rifle as speargun), SteamFire (Flame Staff as trident)
Aureate: The Maelstrom (Aureate Staff as trident)
Shiverpeak: OooOoo Shooter (Shiverpeak Arquebus as speargun)
Legionnaire: Gladium Trident (Legionnaire Staff as trident)
Tribal: Drakestrike (Tribal Rifle as speargun)
IME, people are a little too hardheaded to let something like being a complete and obvious copy of another set member be good enough to allow membership in a set when the names don’t match. So, while I might include the wiki skins entries in the same sets, I haven’t bothered with adjusting the membership of the actual weapons.