Customer endpoint: Records missed under certain circumstances when using filter with pagination

[David Duus]

Hi

We have active implementations and use cases where we utilize search on customer endpoint with attributes filter.  

Like so: https://integration.visma.net/API/controller/api/v1/customer?attributes=%7B%22TESTATTR%22:%224%22%7D&pageSize=100&pageNumber=1

We also adhere to pagination since that is mandatory. 

The problem we are experiencing is the following. Under certain circumstances (more on that below) we receive no customer when we should receive customer. 

See the following examples:

# Match found, customer returned, as it should

https://integration.visma.net/API/controller/api/v1/customer?attributes=%7B%22TESTATTR%22:%224%22%7D&pageSize=100&pageNumber=1 

# No match found, no customer retured, but should have

https://integration.visma.net/API/controller/api/v1/customer?attributes=%7B%22TESTATTR%22:%223828598964310%22%7D&pageSize=100&pageNumber=1 

If we tweak the last call and omit pageSize & pageNumber or increase pageSize to 1000, we can see the following different outcome. 

# Match found, customer returned, as it should

https://integration.visma.net/API/controller/api/v1/customer?attributes=%7B%22TESTATTR%22:%223828598964310%22%7D 

# Match found, customer returned, as it should

https://integration.visma.net/API/controller/api/v1/customer?attributes=%7B%22TESTATTR%22:%223828598964310%22%7D&pageSize=1000&pageNumber=1 

Our conclusion is that there is a conflict in search/filter and pagination where customers (in this case) are missed due to them being later in the register than page size. Missed or at least not correctly returned in response.  The first query returns correctly matched customer since its in the top 100 range. The next that doesn't correctly return is furter down in the 400 range and thus not returned when using page size 100 in this case. 

Since pagination is best practise mandatory we end up in catch 22. Would like feedback on above scenarios and how to proceed. Thanks!

[Yıldırım]

Hello David, 

currently, there is an issue on filtering with Attributes while using Pagination.

The problem is the same as for customer, inventory and supplier endpoints. (Attributes & Pagination) The reason for the error is that the endpoints are implemented with "join" in memory - fetched first all attributes based on the filter and then skipped all inventories, customers and supplier which are not connected to the selected attributes. This operation is done by first getting all inventories, customer or supplier based on pagination information and then joined in memory (instead of joining attributes to the specific tables)

The solution for the problem is to create new endpoints (v2) for inventory , customer and supplier where SQL is used to join attributes.

So we're still waiting for the roadmap to be finalized before announcing when this can be fixed. 

[David Duus]

Hi Yildirim

Ok, thanks for info. Since the solution as to when this is resolved is unknown we have no other option than to disable pagination temporarily. Until your new v2 endpoints are published into production and consumed by us. This problem is causing troubles for our mutual customers at the moment. We have no other option as I see it, or do you have any we should take into consideration?

Thanks

[Florian Haase]

Only way we found to get around is to use the metatags for totalCount and loop through all pages even if they are empty:

expectedPages = (int)Math.Ceiling((decimal)totalCount / maxPageSize);

[David Duus]

Hi Florian

Thanks for your feedback. Will forward this to my colleagues.

[Yıldırım]

Hello, 

Unfortunately there is no workaround at this moment that could resolve this issue. We've also updated the relevant case in our backlog with your feedback accordingly.