SOAP Message with unicode failed

I'm using Perl (v 5.8.8), WWW-Salesforce.com , SOAP::Lite.
When I send query to SFDC API server, I get the correct response.

Code:

SOAP::Serializer::envelope: query SOAP::Data=HASH(0x8424e2c)
SOAP::Data::new: ()
SOAP::Data::new: ()
SOAP::Data::new: ()
SOAP::Data::new: ()
SOAP::Data::new: ()
SOAP::Data::new: ()
SOAP::Transport::HTTP::Client::send_receive: HTTP::Request=HASH(0x89a71e8)
SOAP::Transport::HTTP::Client::send_receive: POST https://na5-api.salesforce.com/services/Soap/c/13.0/460300D70000000JQJ4 HTTP/1.1
Accept: text/xml
Accept: multipart/*
Accept: application/soap
Content-Length: 961
Content-Type: text/xml; charset=utf-8
SOAPAction: ""

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Header>
    <sforce:QueryOptions xmlns:sforce="urn:enterprise.soap.sforce.com">
      <batchSize xsi:type="xsd:int">500</batchSize>
    </sforce:QueryOptions>
    <sforce:SessionHeader xmlns:sforce="urn:enterprise.soap.sforce.com"><sessionId xsi:type="xsd:string">460300D70000000JQJ4!ARgAQD43iHOagg0b0QUjyT23N.A6aM1cVdAghMZLfg5Q21fxIjRB3q8RVxTdkf4dmv0r0nNdtFMVSRjd3.5HEkHTOgsmyDO0</sessionId>
    </sforce:SessionHeader>
  </soap:Header>
  <soap:Body>
    <query xmlns="urn:enterprise.soap.sforce.com">
      <queryString xsi:type="xsd:string">
        Select Id, IsDeleted, MasterRecordId, Name FROM Account WHERE  Name like '%Agentur%' 
      </queryString>
    </query>
  </soap:Body>
</soap:Envelope>
SOAP::Transport::HTTP::Client::send_receive: HTTP::Response=HASH(0x89a6dec)
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 200 OK
Date: Wed, 22 Oct 2008 13:28:58 GMT
Server:
Content-Type: text/xml; charset=utf-8
Client-Date: Wed, 22 Oct 2008 13:29:02 GMT
Client-Peer: 204.235.24.47:80
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /O=VeriSign Trust Network/OU=VeriSign, Inc./OU=VeriSign International Server CA - Class 3/OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
Client-SSL-Cert-Subject: /C=US/ST=California/L=San Francisco/O=Salesforce.com, Inc./OU=Applications/OU=Terms of use at www.verisign.com/rpa (c)00/CN=na5-api.salesforce.com
Client-SSL-Cipher: RC4-MD5
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope>
  <soapenv:Body>
    <queryResponse>
      <result>
        <done>true</done>
        <queryLocator xsi:nil="true"/>
        <records xsi:type="sf:Account">
          <sf:Id>0017000000S7YPAAA3</sf:Id>
          <sf:IsDeleted>false</sf:IsDeleted>
          <sf:Name>Agentur für Arbeit</sf:Name>
        </records>
        <size>1</size>
      </result>
    </queryResponse>
  </soapenv:Body>
</soapenv:Envelope>
SOAP::Deserializer::deserialize: ()
SOAP::Parser::decode: ()
SOAP::SOM::new: ()
SOAP::SOM::DESTROY: ()
SOAP::Lite::DESTROY: ()
SOAP::Deserializer::DESTROY: ()
SOAP::Parser::DESTROY: ()
SOAP::Transport::DESTROY: ()
SOAP::Transport::HTTP::Client::DESTROY: ()
SOAP::Serializer::DESTROY: ()
SOAP::Data::DESTROY: ()

When the queryString contains utf-8 encoded characters, my request message turns out ok, but I'm getting an error from the server.

Code:

SOAP::Serializer::envelope: query SOAP::Data=HASH(0x8424e2c)
SOAP::Data::new: ()
SOAP::Data::new: ()
SOAP::Data::new: ()
SOAP::Data::new: ()
SOAP::Data::new: ()
SOAP::Data::new: ()
SOAP::Transport::HTTP::Client::send_receive: HTTP::Request=HASH(0x89b8610)
SOAP::Transport::HTTP::Client::send_receive: POST https://na5-api.salesforce.com/services/Soap/c/13.0/460300D70000000JQJ4 HTTP/1.1
Accept: text/xml
Accept: multipart/*
Accept: application/soap
Content-Length: 958
Content-Type: text/xml; charset=utf-8
SOAPAction: ""

<?xml version="1.0" encoding="utf-8"?>
  <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <soap:Header>
      <sforce:QueryOptions xmlns:sforce="urn:enterprise.soap.sforce.com">
        <batchSize xsi:type="xsd:int">500</batchSize>
        </sforce:QueryOptions><sforce:SessionHeader xmlns:sforce="urn:enterprise.soap.sforce.com"><sessionId xsi:type="xsd:string">460300D70000000JQJ4!ARgAQD43iHOagg0b0QUjyT23N.A6aM1cVdAghMZLfg5Q21fxIjRB3q8RVxTdkf4dmv0r0nNdtFMVSRjd3.5HEkHTOgsmyDO0</sessionId></sforce:SessionHeader>
    </soap:Header>
    <soap:Body>
      <query xmlns="urn:enterprise.soap.sforce.com">
        <queryString xsi:type="xsd:string">
          Select Id, IsDeleted, MasterRecordId, Name FROM Account WHERE  Name like '%für%' 
        </queryString>
      </query>
    </soap:Body>
  </soap:Envelope>
SOAP::Transport::HTTP::Client::send_receive: HTTP::Response=HASH(0x89b91bc)
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 500 Internal Server Error
Date: Wed, 22 Oct 2008 13:29:06 GMT
Server:
Content-Type: text/xml; charset=utf-8
Client-Date: Wed, 22 Oct 2008 13:29:11 GMT
Client-Peer: 204.235.24.47:80
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /O=VeriSign Trust Network/OU=VeriSign, Inc./OU=VeriSign International Server CA - Class 3/OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
Client-SSL-Cert-Subject: /C=US/ST=California/L=San Francisco/O=Salesforce.com, Inc./OU=Applications/OU=Terms of use at www.verisign.com/rpa (c)00/CN=na5-api.salesforce.com
Client-SSL-Cipher: RC4-MD5
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope>
  <soapenv:Body>
    <soapenv:Fault>
      <faultcode>soapenv:Client</faultcode>
      <faultstring>
        The element type "soap:Envelope" must be terminated by the matching end-tag "</soap:Envelope>".
      </faultstring>
    </soapenv:Fault>
  </soapenv:Body>
</soapenv:Envelope>
SOAP::Deserializer::deserialize: ()
SOAP::Parser::decode: ()
SOAP::SOM::new: ()
The element type "soap:Envelope" must be terminated by the matching end-tag "</soap:Envelope>". at testAPI.pl line 37
SOAP::SOM::DESTROY: ()
SOAP::Lite::DESTROY: ()
SOAP::Deserializer::DESTROY: ()
SOAP::Parser::DESTROY: ()
SOAP::Transport::DESTROY: ()
SOAP::Transport::HTTP::Client::DESTROY: ()
SOAP::Serializer::DESTROY: ()
SOAP::Data::DESTROY: ()

Everything is encoded with utf-8. My orgernization setting from 'describeGlobal' says server expects encoding 'utf-8'. The fault strings says my message is not well formed xml, besides that I noticed there was a '500 Internal Server Error' when I'm getting the response for the query with unicode.

I'm also able to get query results from the server which contains unicode characters.

Help is appreciated.

October 22, 2008
·
Answer
·
Like
0
·
Follow
0

Superfell
I suspect that whatever's working out the content length for the content-length header is counting chars instead of bytes, so the server stops reading the request before its actually finished.

October 22, 2008
·
Like
1
·
Dislike
0

gmac
Thanks Simon,
I tried to use the same query string only varies at one character (one ascii, on unicode). The message headers sent out was correctly counting the content length (i.e. 958 vs 959). Besides, the server shoudn't rely on the content-length when reading messages.

October 22, 2008
·
Like
0
·
Dislike
0

Superfell
Your request is HTTP/1.1 therefore the server is absolutely required to take notice of the content-length header. Sorry i don't have much else to help, except that i've made similar requests from .NET & Java and they work fine.

October 22, 2008
·
Like
1
·
Dislike
0

gmac
Simon,
You're are absolutely right on.
It's actually well documented problem in the SOAP::Lite. The content-length I was getting are produced by SOAP::Lite which was using byte::length() to get the length, but all of that was over written by the libwww (LWP) which uses length().

/usr/local/share/perl/5.8.8/SOAP/Transport/HTTP.pm
Code:
COMPRESS: { my $compressed = !exists $nocompress{$endpoint} && $self->options->{is_compress} && ($self->options->{compress_threshold} || 0) < length $envelope; $envelope = Compress::Zlib::memGzip($envelope) if $compressed; my $original_encoding = $http_request->content_encoding; while (1) { # check cache for redirect $endpoint = $redirect{$endpoint} if exists $redirect{$endpoint}; # check cache for M-POST $method = 'M-POST' if exists $mpost{$endpoint}; # what's this all about— # unfortunately combination of LWP and Perl 5.6.1 and later has bug # in sending multibyte characters. LWP uses length() to calculate # content-length header and starting 5.6.1 length() calculates chars # instead of bytes. 'use bytes' in THIS file doesn't work, because # it's lexically scoped. Unfortunately, content-length we calculate # here doesn't work either, because LWP overwrites it with # content-length it calculates (which is wrong) AND uses length() # during syswrite/sysread, so we are in a bad shape anyway. # # what to do– we calculate proper content-length (using # bytelength() function from SOAP::Utils) and then drop utf8 mark # from string (doing pack with 'C0A*' modifier) if length and # bytelength are not the same my $bytelength = SOAP::Utils::bytelength($envelope); $envelope = pack('C0A*', $envelope) if !$SOAP::Constants::DO_NOT_USE_LWP_LENGTH_HACK && length($envelope) != $bytelength; $http_request->content($envelope); $http_request->protocol('HTTP/1.1'); $http_request ->proxy_authorization_basic($ENV{'HTTP_proxy_user'}, $ENV{'HTTP_proxy_pass'}) if ($ENV{'HTTP_proxy_user'} && $ENV{'HTTP_proxy_pass'});

I'm still not sure why the SOAP::LIte 'Hack' didn't work. And I really don't have a good solution other than completely patch the LWP to use byte::length() in this file
/usr/share/perl5/LWP/Protocol/http.pm
as suggested in this link
http://www.mail-archive.com/libwww@perl.org/msg06097.html

This is definitely not a solution but at least I can work around it for now.

Thanks again for your help!!

October 22, 2008
·
Like
0
·
Dislike
0

You need to sign in to do that.

Need an account? Sign Up

Have an account? Sign In

Dismiss

Browse by Topic

Welcome to Support!

Show

sorted by

SOAP Message with unicode failed

You need to sign in to do that.