StreamSets

StreamSets

Connect with experts and peers to elevate technical expertise, solve problems and share insights.

 View Only

HTTP Client supports data-urlencode values to be passed via OAUTH 2

  • 1.  HTTP Client supports data-urlencode values to be passed via OAUTH 2

    Posted Tue July 11, 2023 05:19 AM

    ISSUE:

    While passing data-urlencode values via OAUTH 2 in HTTP client, We were getting error message-”HTTP_01 - Error fetching resource. HTTP-Status: 400 Reason: Bad Request”.

    The endpoint is microsoftonline oauth2. When we run below query in postman it works.


    curl --location --request GET 'https://login.microsoftonline.com/<xxx>/oauth2/v2.0/token \
    --header 'Content-Type: application/x-www-form-urlencoded' \
    --data-urlencode 'grant_type=client_credentials' \
    --data-urlencode 'client_id=<stripped>' \
    --data-urlencode 'client_secret=<stripped>' \
    --data-urlencode 'scope=https://xxx.com/.default'
     

     

    but this is not working when we do this in HTTP client with below configuration.

    Configuration:

    Error Message:

    HTTP_01 HTTP_01 - Error fetching resource. HTTP-Status: 400 Reason: Bad Request 

     

    Resolution:

    1. Do add below property in HTTP 
      1. accept:*/*

         

         

    2. We might be getting few error which can be fixed by following below process
      1. Error-"HTTP_00 - Cannot parse record. HTTP-Status:200
        cause:Reason:com.streamsets.pipeline.api.ext.io.OverrunException:Reader exceeded the read limit '1048576" then
        Solution:
        1. Update "parser.limit=104857600" on sdc.properties.Do remove the leading #
        2. restart the sdc.
      2. Error-Max object length exceeded from 4096.
        Solution:Update the Max object length to 2147483647 in the data format tab under HTTP stage section.