AI connection tool

Extensions and Python Add-ons Visual Components Add-on Samples
1

Does the software have a similar AI connection tool?

a
a1308×643 64.7 KB

2

No, I believe that is some personal .NET plugin made by @chungmin which uses online services.

3

Yeah, you would need NET plugin to communicate with AI server, and you also need to purchase credits for using AI API, here are some snippet codes

  1. Text only
public void AskChatGPT()
{
    var task = Task.Run(() => execute_AskChatGPT(Background + "\n" + Question));
    task.Wait();
    Answer = task.Result;
}

public async Task<string> execute_AskChatGPT(string prompt)
{
    using (var httpClient = new HttpClient())
    {
        httpClient.DefaultRequestHeaders.Add("Authorization", $"Bearer {ChatGPTAPIKey}");

        var request_data = new
        {
            model = "gpt-4o-mini",
            messages = new[]
            {
                new
                {
                    role = "user",
                    content = prompt
                }
            },

            max_tokens = 3000,
            temperature = 0.0
        };

        var json_content = JsonConvert.SerializeObject(request_data);
        var http_content = new StringContent(json_content, Encoding.UTF8, "application/json");

        var response = await httpClient.PostAsync(new Uri("https://api.openai.com/v1/chat/completions"), http_content);
        string result = await response.Content.ReadAsStringAsync();

        if (response.IsSuccessStatusCode)
        {
            dynamic json_response = JsonConvert.DeserializeObject(result);
            return json_response.choices[0].message.content.ToString();
        }
        else
        {
            throw new Exception($"API error: {result}");
        }
    }
}
  1. With current view
public void AskChatGPTWithCurrentView()
{    
    var task = Task.Run(() => execute_AskChatGPTWithCurrentView(Background + "\n" + Question));
    task.Wait();
    Answer = task.Result;    
}

public async Task<string> execute_AskChatGPTWithCurrentView(string prompt)
{
    var buffer = app.SurfaceToBuffer(ImageFormat.JPG, 1920, 1080);
    var base64Image = Convert.ToBase64String(buffer);

    using (var httpClient = new HttpClient())
    {
        httpClient.DefaultRequestHeaders.Add("Authorization", $"Bearer {ChatGPTAPIKey}");

        var request_data = new
        {
            model = "gpt-4o",
            messages = new[]
            {
                new
                {
                    role = "user",
                    content = new object[]
                    {
                        new { type = "text", text = prompt },
                        new { type = "image_url", image_url = new { url = $"data:image/jpeg;base64,{base64Image}" } }
                    }
                }
            },

            max_tokens = 3000,
            temperature = 0.0
        };

        var json_content = JsonConvert.SerializeObject(request_data);
        var http_content = new StringContent(json_content, Encoding.UTF8, "application/json");

        var response = await httpClient.PostAsync(new Uri("https://api.openai.com/v1/chat/completions"), http_content);
        string result = await response.Content.ReadAsStringAsync();

        if (response.IsSuccessStatusCode)
        {
            dynamic json_response = JsonConvert.DeserializeObject(result);
            return json_response.choices[0].message.content.ToString();
        }
        else
        {
            throw new Exception($"API error: {result}");
        }
    }
}

The personal AI API does not have a database available, so we need to use prompts to explain to the AI how to generate a Python script for VC.

Here is an example for text only prompt

1.
from vcScript import *
from vcHelpers.Robot2 import *

app = getApplication()

def OnRun():
    # Your answer will be used here

2. Only reply in Python script, no extra characters and comments
3. When I ask robot to drive J1 = 180.0, reply robot.driveJoints(J1 = 180.0)
4. When I ask robot to pick cylinder, reply robot.pick(app.findComponent(""cylinder""))
5. When I ask robot to place on block, reply robot.place(app.findcomponent(""block""))

Then you can start stating your requirements and ask the AI to generate the script, extract the script from the AI’s answer, create a dummy component to execute it

2 Likes