Page consistency
Leablo.2651:
With respect to the Q&A in this thread: https://forum-en.guildwars2.com/forum/community/api/Order-of-ids-returned-v2-commerce-listings/first#post4445386
My understanding according to the above is that a specific page should consistently return the same set (if not order) of values. But this was not my experience between the HoT launch and now.
It seems that “missing” items, once discovered, are inserted into the paging sequence in an order other than discovery (presumably ID sort). As there were quite a few “unknown” items at launch, and I believe many hundreds of items in total, it was necessary to re-check those pages for new items. I’m not sure if the possibility of missing items was taken into consideration during the previous thread’s discussion, or whether the way HoT items were handled is something new or unintended, or if it’s a problem with my code.
Lawton Campbell.8517:
As context — the API frontend keeps a list of all ids for a given resource, sorted by id (basically what’s emitted when you request e.g. /v2/specializations). When a page is requested, it’s sliced out of the sorted id list, then each resource is fetched asynchronously from a backend server. For ?ids, it just uses the id list passed in. The asynchronous bit causes the output order to get shuffled a bit (which is still on the backlog to fix).
Ids can basically be added in anywhere — our content does not guarantee a monotonically incrementing id. Because of that, only requesting pages means that you’ll always have to do a full re-fetch of the entire dataset to find all resources (and the contents will shift around on the pages). For finding new stuff, the recommended approach is to just grab the list of all ids, diff against the set you’ve already got, then request what’s missing from your local set via ?ids. You’ll probably also want to re-fetch everything whenever a new release comes out because sometimes old items are updated.