AI Feature Integration Guide

This document provides a detailed guide on how to use Supabase Edge Function to call AI APIs in EasyAppSwiftUI, using receipt analysis functionality as an example to demonstrate the complete implementation process.

Why Choose Supabase Edge Function?

Using Supabase Edge Function as an intermediate layer for AI services offers the following significant advantages:

Cost Savings

No additional server maintenance costs, pay-as-you-go model

API Security

API Keys are securely stored on the server side, avoiding client-side packet capture risks

User Management

Easy access to user information for permission control and data management

Security Reminder

Strongly Recommended: Store API Keys on the server side rather than the client side. Network requests from mobile apps are easily intercepted by packet capture tools, leading to API Key leakage.

Want to learn more about Supabase Edge Function? Please refer to the official documentation:

Supabase Edge Functions Official Guide

For more information on how to integrate Supabase Edge Function, please refer to:

Supabase Edge Function

Introduction to Supabase Edge Function

Combined with In-App Purchases

We can combine with in-app purchases to provide different AI analysis counts, such as 10 free, 100 paid, 1000 paid, etc. This feature is currently under development, stay tuned.

System Architecture Overview

The AI feature adopts the following architecture design:

SwiftUI App → Supabase Storage → Supabase Edge Function → OpenAI API → Streaming Response → SwiftUI

Demo

Core Feature Characteristics

Intelligent Receipt Analysis

Image Upload: Support selecting receipt images from photo album or camera
AI Recognition: Use OpenAI multimodal model to extract receipt information
Real-time Streaming Response: Display analysis process to enhance user experience
Structured Data: Automatically parse into standardized JSON format
Data Storage: Analysis results saved to Supabase database

Technical Implementation Details

1. Data Model Design

First, define the structured model for receipt data:

struct ReceiptModel: Identifiable, Codable, Equatable {
    var id = UUID()
    var userID: String?
    let restaurantInfo: RestaurantInfo
    let foodItems: [FoodItem]
    let pricingSummary: PricingSummary
    let paymentInfo: PaymentInfo
    let receiptMetadata: ReceiptMetadata
}

struct RestaurantInfo: Codable {
    let name: String?
    let address: String?
    let phone: String?
    let date: String?
    let time: String?
}

struct FoodItem: Codable, Identifiable {
    var id = UUID()
    let name: String?
    let unitPrice: Double?
    let quantity: Int?
    let totalPrice: Double?
}

2. SwiftUI Frontend Implementation

Image Upload Functionality

func uploadImage(selectedImage: UIImage) async throws -> String {
    isUploading = true
    defer { isUploading = false }
    
    guard let imageData = selectedImage.jpegData(compressionQuality: 0.8) else { 
        return "" 
    }

    let imageName = UUID().uuidString
    let imagePath = "\(imageName).jpeg"

    return try await supabase.request {
        let response = try await supabase.storage
            .from("receipt-analysis")
            .upload(
                imagePath,
                data: imageData,
                options: FileOptions(contentType: "image/jpeg")
            )
        return response.fullPath
    }
}

AI Analysis Call

func analyzeReceipt(imagePath: String) async {
    isAnalyzing = true
    analysisContent = ""
    parsedReceipt = nil

    let parameters = ["imgPath": imagePath]

    do {
        try await supabase.callStreamingFunction(
            functionName: "receipt-analysis",
            parameters: parameters,
            onContent: { [weak self] (content: String) in
                guard let self = self else { return }
                self.analysisContent += content
            },
            onCompletion: { [weak self] in
                guard let self = self else { return }
                self.isAnalyzing = false
                self.parseAnalysisContent()
            }
        )
    } catch {
        isAnalyzing = false
        state = .result(.failure(error))
    }
}

Function Entry and Authentication

Deno.serve(async (req) => {
    const userAuth = await getUserAuth(req);
    if (!userAuth) {
        return new Response(JSON.stringify({ error: "Unauthorized" }), {
            status: 401,
        });
    }
    
    return responseAIModelStream(req);
});

async function getUserAuth(req: Request) {
    const supabaseClient = getSupabaseClient(req);
    const authHeader = req.headers.get("Authorization");
    if (!authHeader) return null;
    
    const token = authHeader.replace("Bearer ", "");
    const { data } = await supabaseClient.auth.getUser(token);
    return data?.user || null;
}

AI Model Call and Streaming Response

async function responseAIModelStream(req: Request) {
    const { imgPath } = (await req.json()) as ReceiptAnalysisRequest;
    
    // Get public URL of the image for AI analysis
    const imageUrl = getImageUrl(req, imgPath);
    console.log("Processing image:", imageUrl);

    const stream = await openAIClient.chat.completions.create({
        model: "qwen-vl-max-latest",
        response_format: { type: "json_object" },
        messages: [
            {
                role: "system",
                content: "You are a professional restaurant receipt analysis expert..."
            },
            {
                role: "user",
                content: [
                    {
                        type: "image_url",
                        image_url: { url: imageUrl }  // Use dynamically generated image URL
                    },
                    {
                        type: "text",
                        text: `Analyze this receipt and return JSON with fields:
                        - restaurant_info
                        - food_items
                        - pricing_summary
                        - payment_info
                        - receipt_metadata`
                    }
                ]
            }
        ],
        stream: true,
    });

    const readableStream = new ReadableStream({
        start(controller) {
            (async () => {
                try {
                    for await (const chunk of stream) {
                        const content = chunk.choices[0]?.delta?.content || "";
                        if (content) {
                            const encoder = new TextEncoder();
                            controller.enqueue(
                                encoder.encode(`data: ${JSON.stringify({ content })}\n\n`)
                            );
                        }
                    }
                    const encoder = new TextEncoder();
                    controller.enqueue(encoder.encode("data: [DONE]\n\n"));
                } catch (error) {
                    controller.error(error);
                } finally {
                    controller.close();
                }
            })();
        },
    });

    return new Response(readableStream, {
        headers: {
            "Content-Type": "text/event-stream",
            "Cache-Control": "no-cache",
            Connection: "keep-alive",
        },
    });
}

3. Data Parsing and Storage

JSON Parsing

func parseAnalysisContent() {
    guard !analysisContent.isEmpty else {
        state = .result(.failure(ParsingError.emptyAnalysisContent))
        return
    }

    let cleanedContent = extractJSON(from: analysisContent)
    
    guard let jsonData = cleanedContent.data(using: .utf8) else {
        state = .result(.failure(ParsingError.invalidJSON))
        return
    }

    do {
        let decoder = JSONDecoder()
        let receipt = try decoder.decode(ReceiptModel.self, from: jsonData)
        self.parsedReceipt = receipt
    } catch {
        state = .result(.failure(error))
    }
}

private func extractJSON(from content: String) -> String {
    guard let firstBrace = content.firstIndex(of: "{"),
          let lastBrace = content.lastIndex(of: "}") else {
        return content
    }
    return String(content[firstBrace...lastBrace])
}

Database Storage

func saveReceiptAnalysisResult(receipt: ReceiptModel) async {
    isSaving = true
    defer { isSaving = false }

    guard let user = supabase.auth.currentUser else { return }

    let saveReceipt = ReceiptModel(
        userID: user.id.uuidString,
        restaurantInfo: receipt.restaurantInfo,
        foodItems: receipt.foodItems,
        pricingSummary: receipt.pricingSummary,
        paymentInfo: receipt.paymentInfo,
        receiptMetadata: receipt.receiptMetadata
    )

    do {
        try await supabase.request {
            try await supabase.client.from("receipt_analysis_results")
                .insert(saveReceipt)
                .select()
                .single()
                .execute()
        }
    } catch {
        state = .result(.failure(error))
    }
}

Configuration Requirements

⚠️ Important Reminder: Please ensure to add the following content to the .gitignore file in your project root directory to avoid sensitive information leakage:

# Environment variable files
.env
.env.local
.env.production
.env.*.local

# Supabase
supabase/.branches
supabase/.temp

1. Supabase Configuration

Environment Variable Setup

Create a .env file in your Supabase project root directory and configure the following environment variables:

# .env file configuration example

# OpenAI API configuration (using Alibaba Cloud Tongyi Qianwen API here)
OPENAI_API_KEY=your_dashscope_api_key_here

# Supabase project configuration
SUPABASE_URL=https://your-project-ref.supabase.co
SUPABASE_SERVICE_ROLE_KEY=your_service_role_key_here

Methods to obtain configuration information:

OPENAI_API_KEY:
- Current code uses Alibaba Cloud Tongyi Qianwen API
- Visit Alibaba Cloud Lingji Platform
- Register an account and create an API Key
- Can also be replaced with standard OpenAI API Key
SUPABASE_URL:
- Login to Supabase
- Select your project
- Find Project URL in Project Settings → Data API page
- Find Service Anon Key in Project Settings → API Keys page

2. Storage Bucket Setup

Create a receipt-analysis storage bucket to store receipt images with appropriate access permissions:

-- Create receipt analysis storage bucket
INSERT INTO storage.buckets (id, name, public) 
VALUES ('receipt-analysis', 'receipt-analysis', true);

-- Set access policy: Allow authenticated users to upload
CREATE POLICY "Allow authenticated users to upload receipts" 
ON storage.objects FOR INSERT 
WITH CHECK (bucket_id = 'receipt-analysis' AND auth.role() = 'authenticated');

-- Set access policy: Allow authenticated users to view their own uploaded files
CREATE POLICY "Allow users to view their own receipts" 
ON storage.objects FOR SELECT 
USING (bucket_id = 'receipt-analysis' AND auth.role() = 'authenticated');

-- Set access policy: Allow users to delete their own files
CREATE POLICY "Allow users to delete their own receipts" 
ON storage.objects FOR DELETE 
USING (bucket_id = 'receipt-analysis' AND auth.role() = 'authenticated');

Storage Bucket Configuration Notes:

public = true: Allow access via public URL (for AI analysis)
Access policies ensure only authenticated users can upload, view, and delete files
Image files automatically generate UUID filenames to avoid naming conflicts

3. Database Table Structure

Complete receipt analysis results table structure:

-- Create receipt analysis results table
CREATE TABLE public.receipt_analysis_results (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id UUID NOT NULL REFERENCES auth.users(id),
    restaurant_info JSONB,
    food_items JSONB,
    pricing_summary JSONB,
    payment_info JSONB,
    receipt_metadata JSONB,
    raw_response JSONB,           -- Store original AI response for debugging
    img_path TEXT,                -- Image storage path
    created_at TIMESTAMPTZ DEFAULT now(),
    updated_at TIMESTAMPTZ DEFAULT now()
);

-- Enable row-level security policies
ALTER TABLE public.receipt_analysis_results ENABLE ROW LEVEL SECURITY;

-- Create access policy: Users can only access their own data
CREATE POLICY "Users can access their own receipt analysis results"
    ON public.receipt_analysis_results
    FOR ALL
    USING (auth.uid() = user_id);

Table Structure Notes:

id: Primary key, auto-generated UUID
user_id: User ID, linked to authentication user table
restaurant_info: Restaurant information (name, address, phone, date, time)
food_items: Food items array (name, unit price, quantity, total price)
pricing_summary: Price summary (subtotal, discount, tax, service fee, total)
payment_info: Payment information (payment method, payment amount, change)
receipt_metadata: Receipt metadata (receipt number, cashier, table number)
raw_response: Original AI response data for debugging and auditing
img_path: Receipt image path in storage
created_at/updated_at: Record creation and update times

Best Practices

Security

Environment Variable Management:
- Do not commit .env files to version control systems
User Authentication: All API calls require valid authentication tokens
Data Validation: Validate format and size of uploaded images
Error Handling: Provide friendly error prompts and retry mechanisms
Row-Level Security: Ensure users can only access their own analysis results

Extended Applications

Based on this architecture, you can easily extend more AI features:

Document Recognition: Extract information from ID cards, passports, and other documents
Text Translation: Multi-language real-time translation functionality
Image Analysis: Product recognition, scene analysis, etc.
Voice Processing: Speech-to-text, etc.

This architecture provides a flexible, scalable AI integration solution that allows you to quickly build various AI application features.

AI Features