Problem uploading an attachment to produt (product image)

[Joonas_Virtasoft]

Hello,

I have a problem uploading a product image, as an attachment, for a product via the Visma.NET API. I'm using C++ with Qt, and trying to replicate the request example (written in C#) from here: https://community.visma.com/t5/Knowledge-base-in-Developers/How-to-POST-an-attachment-via-Visma-net-...

I've came with a solution like this:

...
httpRequest req;
this->initRequest(&req, "controller/api/v1/inventory/"+itemID+"/attachment");

QHttpMultiPart *multipart = req.initMultipart(QHttpMultiPart::FormDataType);

QHttpPart imagePart;
imagePart.setHeader(QNetworkRequest::ContentTypeHeader, QVariant("application/octet-stream"));
imagePart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"file\"; filename=\"" + fileName + "\""));
imagePart.setBody(fileData);

multipart->append(imagePart);

req.waitForData(httpRequest::POST);
...

Few notes:

- fileData variable in this scope is a QByteArray, containing the data of the image file.

- httpRequest class is one made by me, to handle different kind of HTTP Requests.

- initRequest initializes the request by adding necessary authentication headers etc. to the request. This part is confirmed working on other endpoints which work fine.

The API returns with a 400 - Bad request, with response:

{"ExceptionType":"IPPException","ExceptionMessage":"","ExceptionFaultCode":"5520","ExceptionMessageID":"5520_09efa0b5-ac3b-48f8-a8c7-f9095f86a746","ExceptionDetails":""}

I feel like I've tried everything... Does anyone have a working, raw request they could show so I could try and mimic that?

[Yıldırım]

As far as I know,  "ContentDispositionHeaderValue.FileName " is using MIME encoding for non-ascii characters what makes the property value encoded to Base64. 

filename="=?utf-8?B?w5bDlsOWLnBuZw==?="

This is not supported.

Based on  RFC 2047  An 'encoded-word' MUST NOT be used in parameter of a MIME Content-Type or Content-Disposition field, or in any structured field body except within a 'comment' or 'phrase'

Consequently, if the file names contain "non-ASCII" characters, only those characters should be encoded with "ISO-8859-1"  &  "UTF-8" using  RFC 2231

---------------------------acebdf13572468
Content-Disposition: form-data; name="fieldNameHere"; filename=utf-8''%C3%96%C3%96%C3%96.png
Content-Type: image/jpeg

<@INCLUDE *C:...path\ÖÖÖ.png*@>
---------------------------acebdf13572468--

[adrianm]

@Joonas_Virtasoft, sorry for hijacking your thread. I hope support will help you with your issue.

@Yıldırım, thank you for your reply. Trying to summarize what you are saying here.

* "filename*" is not supported (400 response)

* "filename" does not support MIME-encoding (403 response with html)

* "filename" supports "*" encoding but only utf-8 and iso-8859-1

I can't find anything in the RFC:s supporting "*" encoding outside "*" fields but it doesn't matter if that is what Visma.Net supports.

(Strange that mime-encoding worked earlier but not anymore)

But when testing it doesn't work that way. The literals "utf-8''" & "iso-8859-1''" are ignored and just seen as part of the filename. You can see this if you download the file in the GUI.

I would document it as

* "filename" must always be encoded in utf-8 with normal url encoding for characters outside ASCII.

abc.png => filename=abc.png

ÅÄÖ.png => filename=%C3%85%C3%84%C3%96.png

Then it works in both the API and in the GUI but there is a display bug (it opens and downloads correct) which should be reported and fixed.

Just to continue down the rabbit hole I tried to download the uploaded attachment which worked a bit strange. As far as I can tell it works like this:

1. If the filename contain only ascii characters it is returned as-is, "filename=abc.png"

2. If the filename contain characters outside ascii but can be represented in iso-8859-1 it is returned as "filename=iso-8859-1''%C5%C4%D6.png"

3. Otherwise it is returned as utf-8 without prefix, "filename=%C3%85%C3%84%C7%BE.png"

[Yıldırım]

That is true, in conclusion we're seeing that RFC 2231 is partially followed in Attachment related operations, so that, we've informed the development team to review those endpoints. Thank you for sharing your insights @adrianm 

[Joonas_Virtasoft]

I finally had time to continue with this. I read through all the answers and tried to pinpoint my issue. I changed my endpoint to 

controller/api/v1/customerinvoice/<invoiceid>/attachment

 so my testing would be on par with the instructions here, even though my end goal is to add an attachment to a product.

But, I'm still facing the same issue. My problem was not with non-ASCII characters in filename, as I don't have them in the filename, but if I understood correctly from Yildirims post the filename parameter should NOT be in the Content-Disposition header at all, but as a dedicated header. So I tried this:

Headers:

Content-Type: multipart/form-data; boundary=-------------------------acebdf13572468
ipp-application-type: Visma.net Financials
Authorization: Bearer <token>
ipp-company-id: <companyId>
Host: integration.visma.net
Content-Length: 288538 (this is the filesize of Lippis.png)

Body:

-------------------------acebdf13572468
Content-Disposition: form-data; name="file"; 
filename="Lippis.png"
Content-Type: image/png

file data here
-------------------------acebdf13572468

But I still keep getting:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>

If I leave this out:

Content-Type: multipart/form-data; boundary=-------------------------acebdf13572468

so it defaults to:

application/json

I get:

{"ExceptionType":"IPPException","ExceptionMessage":"","ExceptionFaultCode":"5520","ExceptionMessageID":"5520_6685f9d9-d7e6-4e7d-98e7-800638d96d3a","ExceptionDetails":""}

I know there is something wrong with my request, because the examples above a proved to be working by you, but I just can't wrap my head around it... I've tried numerous ways and played around with the Content-Disposition header and Content-Types, to see if they make a difference but nothing... I'm starting to think there might be something wrong with the way Qt creates the Multipart data (how it inserts the file data to the request), because that's pretty much the only thing I don't control in the request, at the moment.

[Yıldırım]

Hello Joonas, we'd recommend you trying this operation in any web debugger to figure out what might be wrong in your code. Unfortunately we're unable to help you with your code snippet written in C++ , Integration-wise code debugging is out of Visma.net ERP API support policy. (Visma.Net API International Support Policy)

Unfortunately I also don't have enough knowledge about the mentioned class in C++ therefore I'm unable to suggest about what should be reviewed within the code in particular. There might be other integrators who are using this, and they could also contribute.

[Joonas_Virtasoft]

Yes I understand, thank you and everyone else for the examples given so far. I will try to debug this and see what is the culprit in my code that does something different.

[Yıldırım]

Topic will remain open so in case I came across with anything that might be helpful, I'll contact you. Thank you for your understanding 🙂 

[Joonas_Virtasoft]

Thank you, I will try again with these instructions! I'll get back to this.

[adrianm]

Uploading attachments is very sensitive and some standard things are not supported.

This works for me

POST: https://integration.visma.net/API/controller/api/v1/customerinvoice/000005/attachment
{
Accept: application/json
Authorization: <hidden>
ipp-application-type: Visma.net Financials
ipp-company-id: ...
Content-Type: multipart/form-data; boundary="144b149f-2756-4594-b45d-76a88ae8de9b"
Content-Length: 19324
}
--144b149f-2756-4594-b45d-76a88ae8de9b
Content-Type: application/octet-stream
Content-Disposition: form-data; name=file; filename=abc.png

<content here>

--144b149f-2756-4594-b45d-76a88ae8de9b--

But if the filename contain non-ascii it is unclear what is supported.

Don't remember but in my code I have comment saying that "filename*" didn't work in 8.12 but encoding filename did (i.e. filename="=?utf-8?B?w4XDhMOWw7gucG5n?="). When I tried now it seems to be the other way around.

[Yıldırım]

Visma.net ERP API - Attachment endpoints don't support RFC 5987 content disposition's  "Filename*" parameter , therefore, returns 400 <Your browser sent a request that this server could not understand> when "filename*" is used.

• However, "non-ASCII" characters can be directly set in the "filename" and they'll work as intended without any need of encoding, Attachment Endpoint transmits characters outside the ISO-8859-1 / UTF 8 and register the file/s in the Visma.net Financials.
• If encoding is preferred, this can still be done by using "Filename" instead of "filename*" with either iso-8859-1'' or utf-8'' 

  ---------------------------acebdf13572468
  Content-Disposition: form-data; name="fieldNameHere"; filename=utf-8''%E6%B5%8B%20%E8%AF%95.png
  Content-Type: image/png
  
  <@INCLUDE *C:\...path...\测 试.png*@>
  ---------------------------acebdf13572468--

• 
• 

Request Headers

Content-Type: multipart/form-data; boundary=-------------------------acebdf13572468
ipp-application-type: Visma.net Financials
Authorization: <token>
ipp-company-id: <companyId>
Host: integration.visma.net
Content-Length: <".filesize($filename))>

Request Body

---------------------------acebdf13572468
Content-Disposition: form-data; name="fieldNameHere"; 
filename="foo-ä-©-¥-₣-Æ..png"
Content-Type: image/png

<@INCLUDE *C:\...path...\foo-ä-©-¥-₣-Æ..png*@>
---------------------------acebdf13572468-

Output:

-------------------
"RFC 2183 indicates that such headers should be encoded according to RFC 2184, which was obsoleted by RFC 2231"
RFC 2231MIME Value and Encoded Word Extensions
-------------------

[adrianm]

Thanks @Yıldırım, very good information. Should be in a FAQ or something.

However, I can't get the encoding to work.

(The standard .Net class "ContentDispositionHeaderValue" automatically encodes everything outside ascii)

filename="=?utf-8?B?w5bDlsOWLnBuZw==?="

returns this strange response

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /API/controller/api/v1/customerinvoice/000005/attachment
on this server.</p>
</body></html>

(a non-encoded filename on the same endpoint works fine)

I know this has worked before because I have it as a unit test.

[Yıldırım]

Thanks @adrianm ! We can prepare a separate article after clarifying the issue you're facing while using that

<p>You don't have permission to access /API/controller/api/v1/customerinvoice/000005/attachment
on this server

Have you tried the same operation with another user (token generated) ? Can you please share the raw string in this way I can try to test further if this is related to .net class encoding or financials user roles ? 

Thanks

[adrianm]

No, I don't have another user for my test company but the normal error on user role problem is  "Invalid parameter exception: companyInContext" so this is something different.

You have the complete request in my other reply which works. Just replace the filename.