WebSphere Application Server & Liberty

WebSphere Application Server & Liberty

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Losing MIME Content-Type Header...

  • 1.  Losing MIME Content-Type Header...

    Posted Tue February 14, 2017 09:48 AM

    Builds:
    WebSphere Application Server Network Deployment, 8.5.5.8 (Build Number: cf081545.03 / Build Date: 11/12/15)
    InfoSphere Information Server v9.1.2.0-FP2-55/Information Services Director v9.1.2.0-1
    DataStage v9.1.2.0

    I've developed DataStage job and a InfoSphere Information Server's Information Services Director web service to call the job.  The job queries a SQL database and returns the contents of a column containing a performed JSON package to the web service, running under WebSphere Application Server.

    A different set of MIME header fields are received depending on how large the JSON package retrieved is.

    MIME columns returned when JSON package less than 4k (<4096 bytes):

    Content-Language
    Date
    Content-Length
    #status#
    Expires
    Content-Type
    X-Powered-By
    Cache-Control
    Pragma

    MIME columns returned when JSON package 4k or more (>=4096 bytes):

    Content-Language
    Date
    Content-Length
    #status#
    X-Powered-By

    My specific issue is the Content-Type header field is present when the JSON retrieved is 4095 bytes or less, missing where it is 4096 bytes or more.

    I have a feeling this is a WebSphere issue, not a DataStage or ISS one, but being a newbie to WebSphere I'm not sure what to tell our admin to try to resolve it.

    Any help would be greatly appreciated!

    Best regards,

    Stephen H. Oswald
    Developer, Data Integration
    Vistana Signature Experiences
    Orlando, FL



  • 2.  RE: Losing MIME Content-Type Header...

    Posted Tue February 14, 2017 05:48 PM

    Hi Stephen,

      Do you have any error in logs, can you share?

     

     Regards,



  • 3.  RE: Losing MIME Content-Type Header...

    Posted Wed February 15, 2017 07:05 AM


    Gabriel,

    Thank you for your reply.  I've requested our admin send me the logs previous to my 2/14 post on this issue, so hopefully will have them soon.  If something jumps out, I'll send it along.

    Best regards,

    Stephen H. Oswald
    Developer, Data Integration
    Vistana Signature Experiences



  • 4.  RE: Losing MIME Content-Type Header...

    Posted Wed February 15, 2017 04:27 PM

    I had our admin turn on http logging in WebSphere and recycle the server.

    After redeploying and running a couple of test calls on the web service causing the issue, I inspected the following WebSphere logs:

    /opt/IBM/WebSphere/AppServer/profiles/InfoSphere/logs/server1/http_access.log
    /opt/IBM/WebSphere/AppServer/profiles/InfoSphere/logs/server1/http_error.log

    In the http_access.log, I found the entries that reference the web service calls:

    10.48.119.113 - - [15/Feb/2017:15:02:33 -0500] "POST /wisd-rest2/Vistana_ODS/WebPkgOutput/ISD_WebPkgOutputPOST/ HTTP/1.1" 200 4095
    10.48.119.113 - - [15/Feb/2017:15:02:57 -0500] "POST /wisd-rest2/Vistana_ODS/WebPkgOutput/ISD_WebPkgOutputPOST/ HTTP/1.1" 200 4096

    The http_error.log is empty.

    So, not finding anything helpful in these logs.  Are they other logs that I should look at?

    Thanks!

    Stephen H. Oswald
    Developer, Data Integration
    Vistana Signature Experiences



  • 5.  RE: Losing MIME Content-Type Header...

    Posted Thu February 16, 2017 04:04 AM

    Hi Stephen,

      from this logs point of view as you say there is no problem you are getting 200 as html return code.
     
      At /opt/IBM/WebSphere/AppServer/profiles/InfoSphere/logs/server1/ there is a SystemOut.log file you can review it.

      My suggestion to see if there is a problem at WAS level. Enable traces of the webcontainer, you can do dynamically, so you can enable, repeat the issue and disable.
     
      MustGather: Web container and Servlet engine problems in WebSphere Application Server
      http://www-01.ibm.com/support/docview.wss?uid=swg21384592
     
      Review if your trace log is correctly configured and you have space in the device (I use 10 files of 10MB)
     
      Enable at server1 RUNTIME tab the next trace string:
     
      *=info:com.ibm.ws.webcontainer*=all:com.ibm.wsspi.webcontainer*=all:HTTPChannel=all:GenericBNF=all:com.ibm.ws.rrd*=all:com.ibm.ws.ard*=all
     
      Repeat the issue
     
      Disable traces: *=info
     
      Hope this helps. Tell us if you need more support
     
    Regards



  • 6.  RE: Losing MIME Content-Type Header...

    Posted Thu February 16, 2017 04:16 AM

    Stephen,

      Are you having the same problem with different browsers (IE, Firefox, chrome)?

    regards



  • 7.  RE: Losing MIME Content-Type Header...

    Posted Thu February 16, 2017 08:16 AM

    Gabriel,

    I took a look at the SystemOut.log entries this morning, but not seeing anything helpful.

    I'll have my admin turn on the webcontainer trace using your suggestions.

    I'm see the same issue in SoapUI (what I used unit testing) and using the Postman extension in Chrome.

    Thanks again for your replies and continued support on this one.

    Stephen H. Oswald
    Developer, Data Integration
    Vistana Signature Experiences



  • 8.  RE: Losing MIME Content-Type Header...

    Posted Thu February 16, 2017 12:13 PM

    Gabriel,

    I think we might getting somewhere!

    Here are the trace.log entries where the Content-Type MIME header is returned (JSON package size retrieved = 4095 bytes):

    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse getBufferSize size --> 4096 [com.ibm.ws.webcontainer.srt.SRTServletResponse@fe60b550]
    [2/16/17 10:07:58:695 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream init init
    com.ibm.ws.webcontainer.srt.SRTOutputStream@eb718081
    [2/16/17 10:07:58:695 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream init initNewBuffer --> 4096
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse fireOutputStreamRetrievedEvent _firstOutputStreamRetrieval:true ,outputMethodListener:null [com.ibm.ws.webcontainer.srt.SRTServletResponse@fe60b550]
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes < com.ibm.ws.webcontainer.srt.SRTServletResponse getOutputStream RETURN
    [2/16/17 10:07:58:695 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream write write len --> 4095
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse alertFirstWrite entry
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse alertFirstWrite _outWriter: --> java.io.OutputStreamWriter@8f69481c [com.ibm.ws.webcontainer.srt.SRTServletResponse@fe60b550]
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse alertFirstWrite exit
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes > com.ibm.ws.webcontainer.srt.SRTServletResponse setContentType ENTRY type --> application/json [com.ibm.ws.webcontainer.srt.SRTServletResponse@fe60b550]
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse getRequest request=com.ibm.ws.webcontainer.srt.SRTServletRequest@7464abe6 [com.ibm.ws.webcontainer.srt.SRTServletResponse@fe60b550]
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletReq 1 com.ibm.ws.webcontainer.srt.SRTServletRequest getWebAppDispatcherContext
    [2/16/17 10:07:58:695 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream isCommitted isCommitted: false
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse isCommitted responseBuffer isCommitted=false [com.ibm.ws.webcontainer.srt.SRTServletResponse@fe60b550]
    [2/16/17 10:07:58:695 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream isCommitted isCommitted: false
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse setContentType did not get writer, check to see if charset already exists
    [2/16/17 10:07:58:695 EST] 00000703 BNFHeadersImp 3 setHeader(h,s): Content-Type
    [2/16/17 10:07:58:695 EST] 00000703 BNFHeadersImp 1 Adding header [Content-Type] with value [application/json]
    [2/16/17 10:07:58:695 EST] 00000703 SRTServletRes < com.ibm.ws.webcontainer.srt.SRTServletResponse setContentType RETURN type=application/json
    [2/16/17 10:07:58:695 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream isCommitted isCommitted: false

     

    Here are the trace.log entries where the Content-Type MIME header is not returned (JSON package size retrieved = 4096 bytes):

    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse getBufferSize size --> 4096 [com.ibm.ws.webcontainer.srt.SRTServletResponse@43028a22]
    [2/16/17 10:08:04:307 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream init init
    com.ibm.ws.webcontainer.srt.SRTOutputStream@fa0d8044
    [2/16/17 10:08:04:307 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream init initNewBuffer --> 4096
    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse fireOutputStreamRetrievedEvent _firstOutputStreamRetrieval:true ,outputMethodListener:null [com.ibm.ws.webcontainer.srt.SRTServletResponse@43028a22]
    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes < com.ibm.ws.webcontainer.srt.SRTServletResponse getOutputStream RETURN
    [2/16/17 10:08:04:307 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream write write len --> 4096
    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse alertFirstWrite entry
    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse alertFirstWrite _outWriter: --> java.io.OutputStreamWriter@358129cc [com.ibm.ws.webcontainer.srt.SRTServletResponse@43028a22]
    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse alertFirstWrite exit
    [2/16/17 10:08:04:307 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream flushBytes flushBytes
    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse alertFirstFlush entry
    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse alertFirstFlush _outWriter: --> java.io.OutputStreamWriter@358129cc [com.ibm.ws.webcontainer.srt.SRTServletResponse@43028a22]
    [2/16/17 10:08:04:307 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream isCommitted isCommitted: false
    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse isCommitted responseBuffer isCommitted=false [com.ibm.ws.webcontainer.srt.SRTServletResponse@43028a22]
    [2/16/17 10:08:04:307 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream isCommitted isCommitted: false
    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes > com.ibm.ws.webcontainer.srt.SRTServletResponse commit ENTRY [com.ibm.ws.webcontainer.srt.SRTServletResponse@43028a22]
    [2/16/17 10:08:04:307 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream isCommitted isCommitted: false
    [2/16/17 10:08:04:307 EST] 00000703 SRTServletRes 1 com.ibm.ws.webcontainer.srt.SRTServletResponse isCommitted responseBuffer isCommitted=false [com.ibm.ws.webcontainer.srt.SRTServletResponse@43028a22]
    [2/16/17 10:08:04:307 EST] 00000703 util 1 com.ibm.wsspi.webcontainer.util.BufferedServletOutputStream isCommitted isCommitted: false
    [2/16/17 10:08:04:307 EST] 00000703 BNFHeadersImp 3 getHeaderAsString(h): Content-Language [null]
    [2/16/17 10:08:04:307 EST] 00000703 BNFHeadersImp 3 setHeader(h,s): Content-Language
    [2/16/17 10:08:04:307 EST] 00000703 BNFHeadersImp 1 Adding header [Content-Language] with value [en-US]

     

    The first entry in both log blocks to pull in the webcontainer.srt.SRTServletResponse buffer size value of 4096.  If I could locate in the server config where to increase this size, I think this might fix this issue.

    As always, any help would be greatly appreciated!

    Stephen H. Oswald
    Developer, Data Integration
    Vistana Signature Experiences



  • 9.  RE: Losing MIME Content-Type Header...

    Posted Thu February 16, 2017 03:56 PM

    Stephen,

      you have it againts the ropes :-)
     
      I don't find response buffer size parameter :-(
     
      Searching Web container custom properties
      https://www.ibm.com/support/knowledgecenter/SSEQTP_8.5.5/com.ibm.websphere.base.doc/ae/rweb_custom_props.html
     
      I have found that for download large files changing this webcontainer property from "async" to "sync" helps but not sure if is your case.


      com.ibm.ws.webcontainer.channelwritetype sync

      I understand that you are not writing your own code and you are using datastage "tools" or utilities to do the webservice.

      I have search in WebSphere Application Server forum but I haven't find nothing related to change response buffer size but you can ask, you have all the information needed (WAS version and last traces).
      https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000266&ps=25

      The last option is to open a PMR.

      Hope this helps. Tell us if you need more support
     
    Regards



  • 10.  RE: Losing MIME Content-Type Header...

    Posted Thu February 16, 2017 04:35 PM

    Gabriel,

    Thanks for your time and input on this one.  I'll try the async to sync change and see if that helps.

    I might just end up coding this one in something other than DataStage, if the folks consuming this service absolutely have to have the Content-Type header.

    Best regards,

    Stephen H. Oswald
    Developer, Data Integration
    Vistana Signature Experiences