🧩 Go Script Demo

GitHub Repository https://github.com/CafeScraper/GoScirptDemo

Required Files (Located in the Project Root Directory)

├── main.go
├── main
├── input_schema.json
├── README.md
├── go.mod
├── go.sum
└── GoSdk
      ├── sdk.go
      ├── sdk_pd.go
      └── sdk_grpc_pd.go

File Name	Description
`main.go`	Script source code
`main`	Script entry executable (execution entry point), must be named `main`
`input_schema.json`	UI input form configuration
`README.md`	Project documentation
`sdk.go`	Core SDK functionality (located in `GoSdk` directory)
`sdk_pd.go`	Data processing enhancement module (located in `GoSdk` directory)
`sdk_grpc_pd.go`	Network communication module (located in `GoSdk` directory)

Go Scripts Must Be Built into an Executable Before Uploading

set CGO_ENABLED=0
set GOOS=linux
set GOARCH=amd64
go build -o main ./main.go

💡 It is recommended to compress the executable using UPX after building to significantly reduce file size.

⭐ Core SDK Files

📁 File Overview

The following three SDK files are required and must be placed in the root directory of the script:

File Name	Main Function
`sdk.go`	Core SDK functionality
`sdk_pd.go`	Data processing enhancement module
`sdk_grpc_pd.go`	Network communication module

These files together form the script’s toolbox, providing all essential capabilities required for crawler execution and communication with the platform backend.

🔧 Core Feature Usage Guide

1. Environment Parameters – Retrieve Script Input Configuration

When the script starts, configuration parameters (such as target website URLs or search keywords) can be passed in externally. Use the following method to retrieve them:

// Retrieve all input parameters as a JSON string
ctx := context.Background()
inputJSON, _ := cafesdk.Parameter.GetInputJSONString(ctx)

// Example return value:
// {"website": "example.com", "keyword": "technology news"}

Use case:
You can reuse the same script to crawl different websites or datasets simply by changing input parameters, without modifying the code.

2. Runtime Logs – Record Script Execution Process

During execution, logs of different levels can be recorded and displayed in the platform console for monitoring and debugging:

ctx := context.Background()

// Debug information (most detailed, for troubleshooting)
SDK.Log.Debug(ctx, "Connecting to target website...")

// Informational logs (normal workflow)
SDK.Log.Info(ctx, "Successfully retrieved 10 news items")

// Warning logs (non-critical issues)
SDK.Log.Warn(ctx, "Network latency is high, performance may be affected")

// Error logs (execution failures)
SDK.Log.Error(ctx, "Failed to access target website, please check network connection")

Log Levels:

debug: Detailed debugging information (recommended during development)
info: Normal execution flow
warn: Warnings that do not stop execution
error: Critical errors that require attention

3. Result Submission – Sending Data Back to the Platform

After data collection, results must be returned to the platform in two steps.

Step 1: Define Table Headers (Required)

Before pushing data, define the table structure (similar to defining column headers in Excel):

headers := []*cafesdk.TableHeaderItem{
    {
        Label:  "Title",
        Key:    "title",
        Format: "text",
    },
    {
        Label:  "Content",
        Key:    "content",
        Format: "text",
    },
}

ctx := context.Background()
res, err := cafesdk.Result.SetTableHeader(ctx, headers)

Field Description:

Label: Column name displayed to users
Key: Unique identifier used in code
Format: Data type, supported values:
- "text" – string
- "integer" – integer
- "boolean" – true/false
- "array" – list
- "object" – dictionary/object

Step 2: Push Data Row by Row

After defining headers, push the collected data one record at a time:

type result struct {
    Title   string `json:"title"`
    Content string `json:"content"`
}

resultData := []result{
    {Title: "Sample Title 1", Content: "Sample Content 1"},
    {Title: "Sample Title 2", Content: "Sample Content 2"},
}

ctx := context.Background()

for _, datum := range resultData {
    jsonBytes, _ := json.Marshal(datum)

    res, err := cafesdk.Result.PushData(ctx, string(jsonBytes))
    if err != nil {
        cafesdk.Log.Error(ctx, fmt.Sprintf("Failed to push data: %v", err))
        return
    }
    fmt.Printf("PushData Response: %+v\n", res)
}

Important Notes:

The order of setting headers and pushing data does not matter
Keys in pushed data must exactly match the keys defined in headers
Data must be pushed one record at a time
Logging after each push is recommended for traceability

⚠️ Common Issues & Notes

File placement: Ensure all SDK files are located in the script directory
Imports: Use SDK or CafeSDK directly to access SDK functionality
Key consistency: Data keys must exactly match table header keys
Error handling: Always check return values, especially when pushing data

⭐ Script Entry File (`main.go`)

💡 Example Code

// (Code unchanged, original example retained verbatim)

(The complete Go example remains exactly as provided above.)

Automated Data Collection Script: Workflow & Principles

1. Script Overview

This script is an automation Script that works like a digital employee.
It automatically opens target web pages (such as social media sites), extracts required information, and organizes the data into structured tables.

2. How Does It Work?

The entire process can be simplified into four main stages:

Step 1: Receive Instructions (Input Parameters)

Before execution, you provide instructions such as:

Target page URL
Number of records to collect

Step 2: Stealth Preparation (Proxy Network Configuration)

To reliably access overseas or restricted websites, the script automatically configures a secure proxy channel.

Step 3: Automated Execution (Business Logic Processing)

This is the core stage where the script:

Visits target pages
Extracts titles, content, images, and other required data

Step 4: Result Reporting (Data Push & Table Generation)

After collection:

Raw data is converted into standardized formats
Results are saved to the system
Table headers (e.g., “URL”, “Content”) are automatically generated

​🧩 Go Script Demo

​Required Files (Located in the Project Root Directory)

​Go Scripts Must Be Built into an Executable Before Uploading

​⭐ Core SDK Files

​📁 File Overview

​🔧 Core Feature Usage Guide

​1. Environment Parameters – Retrieve Script Input Configuration

​2. Runtime Logs – Record Script Execution Process

​3. Result Submission – Sending Data Back to the Platform

​Step 1: Define Table Headers (Required)

​Step 2: Push Data Row by Row

​⚠️ Common Issues & Notes

​⭐ Script Entry File (main.go)

​💡 Example Code

​Automated Data Collection Script: Workflow & Principles

​1. Script Overview

​2. How Does It Work?

​Step 1: Receive Instructions (Input Parameters)

​Step 2: Stealth Preparation (Proxy Network Configuration)

​Step 3: Automated Execution (Business Logic Processing)

​Step 4: Result Reporting (Data Push & Table Generation)