Spotlights:
Novak Ivanovich
Jan 11, 2024
Introduction:
In the era of rapidly evolving artificial intelligence, the integration of external data sources into custom Generative Pre-trained Transformers (GPTs) has emerged as a pivotal advancement. With the recent launch of the GPT store, a global platform for sharing user-created GPTs, the potential for these models to consume organized information via APIs has significantly expanded. This guide explores the transformative impact of integrating external data sources into custom GPTs, particularly through APIs that are not readily accessible via standard internet searches. Examples of such integrations include accessing proprietary databases, utilizing specialized data feeds, or interfacing with unique enterprise systems.
This article will guide you through the process of enhancing custom GPTs by integrating external data sources. It will cover the use of the Action feature in custom GPTs, a mechanism that allows for seamless interaction with external APIs. Additionally, we will delve into how Actions can be used to push data to various platforms, such as emails or messaging services, and the potential applications of this capability. The guide includes a detailed walkthrough on setting up a custom GPT, building an action, creating a schema based on an API specification, troubleshooting common issues, and setting up API authentication.
Required Materials or Preparations:
Materials List:
Access to the OpenAI platform and the custom GPT feature.
An understanding of OpenAPI specifications for integrating APIs.
Relevant API specifications and access credentials (if necessary for private APIs).
Tools for testing API integration, such as Postman or similar software.
Preparatory Steps:
Identifying the Right API: Choose an API that suits your custom GPT's needs. This could be a public API for general data or a private API for specific, proprietary data. Below are resources where you can find APIs for various purposes and industries:
ProgrammableWeb: An extensive directory offering a diverse range of APIs across different sectors.
RapidAPI: A vast API marketplace with a unified interface for connecting to and managing APIs.
GitHub: Explore open-source projects and collections for free APIs, including curated lists under "Awesome APIs."
Public APIs: A collective list of free APIs for development, categorized into areas like health, sports, news, music, etc.
API List: A straightforward website featuring a compilation of useful APIs for various applications.
Postman API Network: Discover thousands of public APIs on Postman's API collaboration platform.
Reddit and Developer Forums: Engage with developer communities on platforms like Reddit for shared insights and discussions on APIs.
Understanding OpenAPI Specification: Familiarize yourself with the OpenAPI specification, a standard format for describing RESTful APIs, which is crucial for integrating APIs with custom GPTs.
Setting up the Environment: Ensure you have an account on the OpenAI platform and access to the custom GPT feature. Also, prepare any necessary tools for API testing and integration.
Step-by-Step Instructions:
Sequential Steps:
Setting Up a Custom GPT:
Access the OpenAI platform and navigate to the custom GPT section.
Create a new GPT or select an existing one to modify.
Define the purpose and scope of your GPT based on the task it will perform.
Building an Action for Custom GPT:
In the GPT editor, select “Configure” and then “Add Action”.
Define your action by providing an OpenAPI specification. You can either fill in the spec manually or paste in a URL where it is hosted.
Ensure that the endpoints in the OpenAPI specification are correctly defined. These endpoints are now called “functions” and are crucial for the GPT to interact with the API.
Creating a Schema Based on an API Specification:
Identify the API you want to integrate and obtain its OpenAPI specification.
Use the OpenAI platform to create a schema that matches the API's structure and requirements. This schema will guide the GPT on how to interact with the API. .
Using a URL for API Documentation:If you have a URL containing the API documentation, you can enter this URL directly into the GPT configuration. The GPT can read this documentation and generate the schema accordingly.
Using Custom GPT to Generate Schema:
Alternatively, you can provide the instruction and example output to a custom GPT, such as the ActionsGPT for which a button is provided in the schema section of the GPT builder, or similar tools, which will then generate the schema for you.
This process involves the following steps:
Ask the custom GPT what information it needs.
Provide the relevant API documentation or details about the API (this can be a direct copy from a webpage or document describing the API).
The custom GPT will generate a schema that you can use. If there are errors or revisions needed, you can provide these back to the GPT for correction.
Importing from an OpenAPI schema:
Actions for GPTs can also be imported from an existing OpenAPI schema. If you have already built a plugin with an OpenAPI specification, you can use this existing plugin manifest to define actions for your GPT.
Testing and Refining the Schema:
After generating the schema, it's important to test it to ensure it works as expected. You might need to refine the schema based on the test results or feedback from the GPT.
Additional Resources and Tutorials:
For a detailed walkthrough, you can refer to tutorials available online, such as the video tutorial on integrating API with custom GPT Actions.
If errors occur, paste the error to the ActionsGPT and ask the ActionsGPT to generate or fix the schema errors based on the API's specification.
Setting Up API Authentication:
Determine the authentication method required by the API (e.g., API key, OAuth token).
Configure the authentication settings in the OpenAI platform to align with the API’s requirements.
Test the authentication to ensure that the GPT can securely access the API.
Testing the API Integration:
Utilize the test button provided in the OpenAI platform GPT builder to test the API integration.
Ensure that the GPT is correctly sending requests to the API and receiving the expected responses.
Troubleshoot any issues that arise during testing, adjusting the schema or API settings as needed.
Handling API Errors and Troubleshooting:
In case of API errors, consult the GPT to retrieve the code that was submitted to the API and the specific errors returned.
Use this information to troubleshoot and resolve issues, ensuring a smooth integration.
API Key Authentication:
API Key: A unique identifier used to authenticate a client to an API. It's typically a long string of letters and numbers.
Purpose: To verify the identity of the client requesting access to the API and to enforce rate limits on the usage.
Field Description: Enter the API key provided by the API service.
Authentication Type: This field allows you to specify the type of authentication mechanism.
Purpose: To define the method by which the API key will be sent in the API request.
Field Description: Choose "API Key" as the authentication type.
Auth Type: Defines how the API key is included in the API request.
Purpose: To specify whether the API key is included as a query parameter, in the header, or as a client credential.
Field Description: Set the 'Auth Type' to “Basic” or “Custom”.
API Key Authentication: Basic vs. Custom Auth Type
Basic Auth Type:
Description: Basic authentication involves sending a username and password (or in this case, an API key as the username and a blank password) with each request.
Use Case: It's commonly used when the API key is to be passed as either the username or password in the header of an HTTP request. This method is typically used when the API supports this format, which can be more secure than sending the key in the query string.
Example:vbnet GET / HTTP/1.1 Host: example.com Authorization: Basic <base64-encoded-API-key>
When to Use: Basic auth is appropriate for internal server networks or simple verification on open data, where security is not a primary concern.
Custom Auth Type:
Description: Custom authentication allows you to define a unique way to pass the API key, such as a custom header or a specific format not covered by basic authentication.
Use Case: It is used when the API requires a custom or specific method to authenticate requests, such as a custom header name like "X-API-KEY" or when the API key needs to be sent in a specific part of the request not covered by standard methods.
Example: vbnet GET / HTTP/1.1 Host: example.com X-API-KEY: abcdef12345
When to Use: Custom auth is necessary when dealing with APIs that have unique requirements for authentication, such as specific header names or when the API key needs to be sent in a non-standard manner.
In summary, the choice between "Basic" and "Custom" auth types for API key authentication depends on the specific requirements of the API you're integrating with. Basic auth is simpler and can be used for less secure applications, while custom auth provides flexibility to meet unique API requirements and is often used for more secure or specialized integrations.
OAuth Authentication:
Client ID: A public identifier for apps.
Purpose: Used by the authorization server to identify the client.
Field Description: Enter the client ID provided by the OAuth provider.
Client Secret: A secret known only to the application and the authorization server.
Purpose: To authenticate the identity of the application to the authorization server.
Field Description: Enter the client secret provided by the OAuth provider.
Authorization URL: The URL where the application will redirect users to authenticate.
Purpose: To initiate the OAuth authorization process.
Field Description: Enter the URL provided by the OAuth service.
Token URL: The URL that the client will use to obtain the access token.
Purpose: Used to exchange an authorization code for an access token.
Field Description: Enter the token URL provided by the OAuth service.
Scopes: Defines the level of access that the application is requesting.
Purpose: To specify the permissions the application needs.
Field Description: Enter the scopes as defined by the OAuth provider.
Additional Parameters: For some services, additional parameters may be required in the OAuth authorization request.
Purpose: To include extra parameters that may be necessary for the OAuth service.
Field Description: Add any additional parameters required by the OAuth service.
These fields and their descriptions are vital for setting up API Key or OAuth authentication in GPT actions. Understanding each field's purpose ensures a correct and secure integration with the API.
Step-by-Step Guide to Integrating Microsoft Graph with Custom GPT using OAuth:
Step 1: Register Your Application in Azure Active Directory
Log in to Azure Portal: Access your account on the Azure Portal.
Access Azure Entra ID: From the navigation menu, select “Azure Entra ID”.
Register Application: Navigate to “App registrations” and click on “New registration”. Provide a meaningful name for your application.
Redirect URI (optional): Enter the URL where users will be redirected after authentication. This might need updating later.
Step 2: Obtain Application (Client) ID and Directory (Tenant) ID
After registering, note down the “Application (client) ID” and “Directory (tenant) ID” displayed on the app overview page.
Step 3: Create a Client Secret
Navigate to Certificates & Secrets: On your app’s registration page, select “Certificates & secrets”.
New Client Secret: Generate a new client secret and record its value.
Step 4: Set Up OAuth in Your GPT Application
Go to the GPT Creator and navigate to Actions, then create a new action.
Use or modify a Sample OpenAPI.json for OAuth. For example:json
{ "openapi": "3.0.0", "info": { "title": "Microsoft Graph API Example", "version": "1.0" }, "servers": [ { "url": "https://graph.microsoft.com/v1.0" } ], "paths": { "/me": { "get": { "operationId": "getUserProfile", "summary": "Get current user profile", "responses": { "200": { "description": "Successful response", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/UserProfile" } } } } }, "security": [ { "OAuth2": [] } ] } } }, "components": { "schemas": { "UserProfile": { "type": "object", "properties": { "displayName": { "type": "string" }, "mail": { "type": "string" } } } }, "securitySchemes": { "OAuth2": { "type": "oauth2", "flows": { "authorizationCode": { "authorizationUrl": "https://login.microsoftonline.com/{tenant}/oauth2/v2.0/authorize", "tokenUrl": "https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token", "scopes": { "User.Read": "Read user profile" } } } } } } }
Replace {Tenant} with your Directory (tenant) ID.
Add additional scopes as needed.
Step 5: Configure Redirect URI
Refresh the GPT Creator Interface to ensure correct display of the redirect URL.
Copy the Call Back URL and add it to the redirects field in the Authentication area of Azure.
Step 6: Reauthorize and Test
Reauthorize Your Application: Update your app settings and reauthorize.
Test the functionality by executing getUserProfile.
Additional Notes:
Ensure the client secret is kept secure.
Make sure the redirect URI in your OAuth request matches what’s configured in Azure AD.
This guide provides a framework for integrating a custom GPT action with Microsoft Graph using OAuth, which can be adapted to other OAuth-based integrations. Remember, OAuth integrations can be complex and may require adjustments based on the specific API and OAuth service being used.