PaaS4GenAI: Connecting Generative AI (WatsonX) On IBM Cloud Platform From Oracle Integration Cloud

OIC Connections

IBM Cloud connection — REST Invoke Type connection for connecting to URL. No security policy needs to be attached.

WatsonX.ai connection — REST Invoke Type connection for connecting to URL. No security policy needs to be attached.

Get Token from IBM Cloud

Once the above connections are created in OIC, first, we would need to get a token calling a watsonx.ai project. For the same in the OIC integration, use the IBM Cloud Connection (created in Section 3.3.3) and configure a REST endpoint trigger connection for connecting to IBM Cloud by passing the API key to get an access token.

• HTTP Method: POST
• Query Parameters: grant_type, apikey
• HTTP Header Name: Content-Type

Sample Response:

{
  "access_token" : "abc",
  "refresh_token" : "abc",
  "token_type" : "abc",
  "expires_in" : 123,
  "expiration" : 123,
  "scope" : "abc"
}

Mapping:

The access token received in the response of this REST Call would be passed for calling the watsonx.ai project to get a response against an input prompt.

Response Against I/P Prompt From watsonx.ai

Now that we have a token to call the watsonx.ai project to send a prompt and get a response against it, we will use the OIC connection WatsonX.ai connection (Created in Section 3.3.3) to get o/p against the I/P received by the OIC Integration.

• HTTP Method: POST
• Query Parameters: version

Sample Request:

{
  "model_id" : "",
  "input" : "",
  "parameters" : {
    "decoding_method" : "",
    "max_new_tokens" : 123,
    "min_new_tokens" : 456,
    "random_seed" : null,
    "stop_sequences" : [ 0, 1 ],
    "temperature" : 0.0,
    "top_k" : 100,
    "top_p" : 1,
    "repetition_penalty" : 1
  },
  "project_id" : ""
}

HTTP Header Name: Content-Type, Accept, Authorization

Sample Response:

{
  "model_id" : "",
  "created_at" : "",
  "results" : [ {
    "generated_text" : "abcdefghijklmn",
    "generated_token_count" : 300,
    "input_token_count" : 200,
    "stop_reason" : "",
    "seed" : 123456
  } ]
}

Mapping:

Lastly, Map the “generated_text” output element of this step to the output element of the OIC Integration "response."

Error Handling

Standard error handling practices are to be followed.

• Global error handler configured to send any error to the desired list of recipients obtained from OIC lookup. 
• Additionally, a call to watsonx.ai API might be encapsulated with a scope, and both the default handler and API invocation error handler can be configured to perform desired actions in case of any error as per requirement. 

Conclusion

Generative AI and its powerful capabilities offer a huge scope and can be used in different business processes and flows for doing repetitive tasks innovatively by minimizing or nullifying manual tasks to the greatest extent possible.

WatsonX’s powerful AI capabilities and features, like being open, trusted, targeted, and empowering, along with preconfigured and trained models for doing tasks like sentiment analysis, information extraction, classification, etc., provide strong use cases that can be leveraged by clients to understand and evaluate different business parameters and outcomes.

Hybrid cloud solutions and multi-cloud solutions are being used by different industry giants to drive their business processes, strategies, and planning. The use of generative AI is already the trend of the hour. 

Using the solution described in this article, customers who are in the Oracle Cloud platform and using Oracle Integration can connect to IBM WatsonX directly to get important insights from information in hand without the need to write complex code or implement logic for achieving things like extracting important information from data, and many similar use cases.

OIC Integration for connecting to WatsonX for generative AI tasks can be designed to work in real-time (using app-driven orchestration) or configured to run in specific fixed schedules to do tasks automatically (using scheduled orchestration). With this solution, we have established the proof of connecting to generative AI platforms like WatsonX from other cloud platforms like Oracle.

Future Enhancements and Roadmap

Presently, the connection from OIC (Oracle) to WatsonX is a two-step process where, firstly, a token must be fetched from IBM Cloud IAM API, and then that token is used to interact with the watsonx.ai platform for getting an output against an input prompt. There is a scope to simplify the solution further by using a single OAuth2.0 connection and getting client secret credentials.

The solution has been developed to connect with the IBM Generative AI Platform WatsonX. However, it has been researched and validated that connectivity to other Gen AI platforms like ChatGPT can also be established with small changes in the Integration design. Further enhancement can be made to make the solution generic, where the user would be given the flexibility to choose the Gen AI platform by passing an additional I/P parameter to the OIC Integration, and the integration will have capabilities to get responses against provided input by connecting to the chosen Gen AI platform.