Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Vertex AI Preview SDK #12880

Merged
merged 53 commits into from
May 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
d162513
Add prototype for Firebase Vertex AI
andrewheard Feb 5, 2024
8643806
Update generative-ai-swift dependency
andrewheard Feb 5, 2024
eae8c94
Use Swift 5.9 `AccessLevelOnImport`
andrewheard Feb 14, 2024
9c99bd1
Set Vertex AI endpoint in `RequestOptions`
andrewheard Feb 20, 2024
2224fa7
Revert "Use Swift 5.9 `AccessLevelOnImport`"
andrewheard Feb 20, 2024
a0b8827
Add alternative API surface that returns `GenerativeModel` instances
andrewheard Feb 20, 2024
e0e5aed
Add sample apps using Vertex AI
andrewheard Feb 22, 2024
50d49ea
Update generative-ai-swift dependency
andrewheard Feb 22, 2024
08ccd2a
Fix optional VertexAIProvider unwrapping
andrewheard Feb 22, 2024
3fbd4e3
Update `generative-ai-swift` dependency
andrewheard Feb 23, 2024
1bf861c
Prototype and sample fixes
andrewheard Mar 4, 2024
c57ce6a
Fork `google-generative-ai` for Firebase (#12564)
andrewheard Mar 18, 2024
fb1c2aa
Use forked `google-generative-ai` in Vertex AI samples (#12574)
andrewheard Mar 18, 2024
8cb3f20
Add CI workflow for Vertex AI (#12577)
andrewheard Mar 18, 2024
a44eecd
Fork `generative-ai-swift` tests for Vertex AI (#12584)
andrewheard Mar 19, 2024
a13a67b
Add local App Check package to sample targets (#12576)
morganchen12 Mar 19, 2024
ef6bf2b
Use `generative-ai-swift` tests in Vertex AI (#12585)
andrewheard Mar 19, 2024
2fb7d30
Add App Check tests in `GenerativeModelTests` (#12590)
andrewheard Mar 20, 2024
d338ef6
Make `generativeModel` an instance method of `VertexAI` (#12599)
andrewheard Mar 21, 2024
31e606f
Update test JSON payloads with Vertex AI output (#12615)
andrewheard Mar 22, 2024
2a0de38
Build Sample in CI (#12616)
paulb777 Mar 25, 2024
dd81092
Fix `CitationMetadata` parsing in Vertex AI (#12626)
andrewheard Mar 25, 2024
7970faa
Manage location on VertexAI instead of model (#12630)
paulb777 Mar 26, 2024
d3c49cc
VertexAI Component tests; fix instance generation for different regio…
paulb777 Mar 27, 2024
eb8102e
Add function calling support in Vertex AI (#12633)
andrewheard Mar 27, 2024
67413f7
Add `-Preview` suffix to `FirebaseVertexAI` library name (#12641)
andrewheard Mar 27, 2024
c53334e
Fix model name in Vertex AI multi-modal sample (#12645)
andrewheard Mar 27, 2024
44d7a8e
Add `totalBillableCharacters` to `CountTokensResponse` in Vertex AI (…
andrewheard Mar 27, 2024
efc5331
Rename the Vertex AI parameter `region` to `location` (#12656)
andrewheard Mar 28, 2024
e98e29f
Update to prod endpoint in Vertex AI (#12657)
andrewheard Apr 2, 2024
2112b41
[VertexAI] Consolidate to single sample app (#12662)
paulb777 Apr 2, 2024
36a9c87
Add platform tags in `x-goog-api-client` (#12680)
andrewheard Apr 2, 2024
7faa8f7
[vertex-ai] Fix for SwiftFormat 0.53.5 (#12683)
paulb777 Apr 2, 2024
6b3fd9a
Add `functionCalls` accessor to `GenerateContentResponse` (#12690)
andrewheard Apr 3, 2024
7bf61d1
Vertex AI preview versioning (#12691)
paulb777 Apr 4, 2024
c641c5e
Remove `location` from Vertex AI public API (#12706)
andrewheard Apr 4, 2024
e5637c1
Remove `apiVersion` from `RequestOptions` in Vertex AI (#12707)
andrewheard Apr 4, 2024
279e438
Add placeholder `GoogleService-Info.plist` to Vertex AI sample (#12724)
andrewheard Apr 8, 2024
8c2e48b
Handle `totalBillableCharacters` optional decoding in Vertex AI (#12715)
andrewheard Apr 10, 2024
81b185c
Add function calling sample in Vertex AI (#12745)
andrewheard Apr 10, 2024
f6dec4e
Add forced function calling support in Vertex AI (#12748)
andrewheard Apr 11, 2024
f4fa5f5
Add system instruction support in Vertex AI (#12749)
andrewheard Apr 11, 2024
d535198
Remove FirebaseAppCheck module from Vertex AI sample app (#12754)
andrewheard Apr 11, 2024
b77e7a2
Rename GenerativeAISample to VertexAISample (#12755)
andrewheard Apr 11, 2024
14b5af1
Add `usageMetadata` to `GenerateContentResponse` in Vertex AI (#12777)
andrewheard Apr 12, 2024
dea1f7e
Add public API tests for `GenerateContentResponse` in Vertex AI (#12791)
andrewheard Apr 15, 2024
9ba1738
[Vertex AI] Add Swift major version to language tag (#12829)
andrewheard Apr 22, 2024
0e51ec3
[Vertex AI] Update `Citation` decoding to handle optional `startIndex…
andrewheard Apr 22, 2024
c3d84a7
[Vertex AI] Re-added `location` parameter to public API (#12835)
andrewheard Apr 23, 2024
3340c70
[Vertex AI] Remove public `Codable` conformance (#12833)
andrewheard Apr 23, 2024
2351f35
[Vertex AI] Add `location` checks for slashes and whitespace (#12849)
andrewheard Apr 25, 2024
aead392
Cherrypick CI fixes into vertex-ai branch (#12862)
andrewheard Apr 29, 2024
d675a68
[Vertex AI] Integrate with AppCheckInterop (#12856)
andrewheard Apr 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/performance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ jobs:
env:
plist_secret: ${{ secrets.GHASecretsGPGPassphrase1 }}
signin_secret: ${{ secrets.GHASecretsGPGPassphrase1 }}
runs-on: macos-12 # TODO: the legacy ObjC quickstarts don't run with Xcode 15.
runs-on: macos-14
steps:
- uses: actions/checkout@v4
- uses: ruby/setup-ruby@v1
Expand All @@ -90,8 +90,9 @@ jobs:
quickstart-ios/performance/GoogleService-Info.plist "$plist_secret"
- name: Test swift quickstart
run: ([ -z $plist_secret ] || scripts/third_party/travis/retry.sh scripts/test_quickstart.sh Performance true swift)
- name: Test objc quickstart
run: ([ -z $plist_secret ] || scripts/third_party/travis/retry.sh scripts/test_quickstart.sh Performance true)
# TODO: The legacy ObjC quickstarts don't run with Xcode 15, re-able if we get these working.
# - name: Test objc quickstart
# run: ([ -z $plist_secret ] || scripts/third_party/travis/retry.sh scripts/test_quickstart.sh Performance true)

quickstart-ftl-cron-only:
if: github.repository == 'Firebase/firebase-ios-sdk' && github.event_name == 'schedule'
Expand Down
68 changes: 68 additions & 0 deletions .github/workflows/vertexai.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: vertexai

on:
pull_request:
paths:
- 'FirebaseVertexAI**'
- '.github/workflows/vertexai.yml'
- 'Gemfile*'
schedule:
# Run every day at 11pm (PST) - cron uses UTC times
- cron: '0 7 * * *'

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
cancel-in-progress: true

jobs:
spm:
strategy:
matrix:
target: [iOS, macOS, catalyst]
os: [macos-13]
include:
- os: macos-13
xcode: Xcode_15.2
runs-on: ${{ matrix.os }}
env:
FIREBASECI_USE_LATEST_GOOGLEAPPMEASUREMENT: 1
steps:
- uses: actions/checkout@v4
- name: Xcode
run: sudo xcode-select -s /Applications/${{ matrix.xcode }}.app/Contents/Developer
- name: Initialize xcodebuild
run: scripts/setup_spm_tests.sh
- uses: nick-fields/retry@v3
with:
timeout_minutes: 120
max_attempts: 3
retry_on: error
retry_wait_seconds: 120
command: scripts/build.sh FirebaseVertexAIUnit ${{ matrix.target }} spm

sample:
strategy:
matrix:
# Test build with debug and release configs (whether or not DEBUG is set and optimization level)
build: [build]
include:
- os: macos-13
xcode: Xcode_15.0.1
- os: macos-14
xcode: Xcode_15.2
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Xcode
run: sudo xcode-select -s /Applications/${{ matrix.xcode }}.app/Contents/Developer
- name: Initialize xcodebuild
run: xcodebuild -list
- name: Placeholder GoogleService-Info.plist for build testing
run: cp FirebaseCore/Tests/Unit/Resources/GoogleService-Info.plist FirebaseVertexAI/Sample/
- uses: nick-fields/retry@v3
with:
timeout_minutes: 120
max_attempts: 3
retry_on: error
retry_wait_seconds: 120
command: scripts/build.sh VertexSample iOS
1 change: 1 addition & 0 deletions FirebaseCore/Sources/FIRApp.m
Original file line number Diff line number Diff line change
Expand Up @@ -829,6 +829,7 @@ + (void)registerSwiftComponents {
@"FIRSessions" : @"fire-ses",
@"FIRFunctionsComponent" : @"fire-fun",
@"FIRStorageComponent" : @"fire-str",
@"FIRVertexAIComponent" : @"fire-vertex",
};
for (NSString *className in swiftComponents.allKeys) {
Class klass = NSClassFromString(className);
Expand Down
2 changes: 1 addition & 1 deletion FirebaseCore/Sources/FIROptions.m
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ - (NSString *)libraryVersionID {
// The unit tests are set up to catch anything that does not properly convert.
NSString *version = FIRFirebaseVersion();
NSArray *components = [version componentsSeparatedByString:@"."];
NSString *major = [components objectAtIndex:0];
NSString *major = [NSString stringWithFormat:@"%02d", [[components objectAtIndex:0] intValue]];
NSString *minor = [NSString stringWithFormat:@"%02d", [[components objectAtIndex:1] intValue]];
NSString *patch = [NSString stringWithFormat:@"%02d", [[components objectAtIndex:2] intValue]];
kFIRLibraryVersionID = [NSString stringWithFormat:@"%@%@%@000", major, minor, patch];
Expand Down
12 changes: 6 additions & 6 deletions FirebaseCore/Tests/Unit/FIROptionsTest.m
Original file line number Diff line number Diff line change
Expand Up @@ -624,7 +624,7 @@ - (void)testVersionConsistency {
int minor = (versionString[2] - '0') * 10 + versionString[3] - '0';
int patch = (versionString[4] - '0') * 10 + versionString[5] - '0';
NSString *str = [NSString stringWithFormat:@"%d.%d.%d", major, minor, patch];
XCTAssertEqualObjects(str, FIRFirebaseVersion());
XCTAssertTrue([FIRFirebaseVersion() hasPrefix:str]);
}

// Repeat test with more Objective-C.
Expand All @@ -638,11 +638,11 @@ - (void)testVersionConsistency2 {
NSRange major = NSMakeRange(0, 2);
NSRange minor = NSMakeRange(2, 2);
NSRange patch = NSMakeRange(4, 2);
NSString *str =
[NSString stringWithFormat:@"%@.%d.%d", [kFIRLibraryVersionID substringWithRange:major],
[[kFIRLibraryVersionID substringWithRange:minor] intValue],
[[kFIRLibraryVersionID substringWithRange:patch] intValue]];
XCTAssertEqualObjects(str, FIRFirebaseVersion());
NSString *str = [NSString
stringWithFormat:@"%d.%d.%d", [[kFIRLibraryVersionID substringWithRange:major] intValue],
[[kFIRLibraryVersionID substringWithRange:minor] intValue],
[[kFIRLibraryVersionID substringWithRange:patch] intValue]];
XCTAssertTrue([FIRFirebaseVersion() hasPrefix:str]);
}

#pragma mark - Helpers
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"colors" : [
{
"idiom" : "universal"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"images" : [
{
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"info" : {
"author" : "xcode",
"version" : 1
}
}
64 changes: 64 additions & 0 deletions FirebaseVertexAI/Sample/ChatSample/Models/ChatMessage.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import Foundation

enum Participant {
case system
case user
}

struct ChatMessage: Identifiable, Equatable {
let id = UUID().uuidString
var message: String
let participant: Participant
var pending = false

static func pending(participant: Participant) -> ChatMessage {
Self(message: "", participant: participant, pending: true)
}
}

extension ChatMessage {
static var samples: [ChatMessage] = [
.init(message: "Hello. What can I do for you today?", participant: .system),
.init(message: "Show me a simple loop in Swift.", participant: .user),
.init(message: """
Sure, here is a simple loop in Swift:

# Example 1
```
for i in 1...5 {
print("Hello, world!")
}
```

This loop will print the string "Hello, world!" five times. The for loop iterates over a range of numbers,
in this case the numbers from 1 to 5. The variable i is assigned each number in the range, and the code inside the loop is executed.

**Here is another example of a simple loop in Swift:**
```swift
var sum = 0
for i in 1...100 {
sum += i
}
print("The sum of the numbers from 1 to 100 is \\(sum).")
```

This loop calculates the sum of the numbers from 1 to 100. The variable sum is initialized to 0, and then the for loop iterates over the range of numbers from 1 to 100. The variable i is assigned each number in the range, and the value of i is added to the sum variable. After the loop has finished executing, the value of sum is printed to the console.
""", participant: .system),
]

static var sample = samples[0]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"info" : {
"author" : "xcode",
"version" : 1
}
}
127 changes: 127 additions & 0 deletions FirebaseVertexAI/Sample/ChatSample/Screens/ConversationScreen.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import FirebaseVertexAI
import GenerativeAIUIComponents
import SwiftUI

struct ConversationScreen: View {
@EnvironmentObject
var viewModel: ConversationViewModel

@State
private var userPrompt = ""

enum FocusedField: Hashable {
case message
}

@FocusState
var focusedField: FocusedField?

var body: some View {
VStack {
ScrollViewReader { scrollViewProxy in
List {
ForEach(viewModel.messages) { message in
MessageView(message: message)
}
if let error = viewModel.error {
ErrorView(error: error)
.tag("errorView")
}
}
.listStyle(.plain)
.onChange(of: viewModel.messages, perform: { newValue in
if viewModel.hasError {
// wait for a short moment to make sure we can actually scroll to the bottom
DispatchQueue.main.asyncAfter(deadline: .now() + 0.05) {
withAnimation {
scrollViewProxy.scrollTo("errorView", anchor: .bottom)
}
focusedField = .message
}
} else {
guard let lastMessage = viewModel.messages.last else { return }

// wait for a short moment to make sure we can actually scroll to the bottom
DispatchQueue.main.asyncAfter(deadline: .now() + 0.05) {
withAnimation {
scrollViewProxy.scrollTo(lastMessage.id, anchor: .bottom)
}
focusedField = .message
}
}
})
}
InputField("Message...", text: $userPrompt) {
Image(systemName: viewModel.busy ? "stop.circle.fill" : "arrow.up.circle.fill")
.font(.title)
}
.focused($focusedField, equals: .message)
.onSubmit { sendOrStop() }
}
.toolbar {
ToolbarItem(placement: .primaryAction) {
Button(action: newChat) {
Image(systemName: "square.and.pencil")
}
}
}
.navigationTitle("Chat sample")
.onAppear {
focusedField = .message
}
}

private func sendMessage() {
Task {
let prompt = userPrompt
userPrompt = ""
await viewModel.sendMessage(prompt, streaming: true)
}
}

private func sendOrStop() {
if viewModel.busy {
viewModel.stop()
} else {
sendMessage()
}
}

private func newChat() {
viewModel.startNewChat()
}
}

struct ConversationScreen_Previews: PreviewProvider {
struct ContainerView: View {
@StateObject var viewModel = ConversationViewModel()

var body: some View {
ConversationScreen()
.environmentObject(viewModel)
.onAppear {
viewModel.messages = ChatMessage.samples
}
}
}

static var previews: some View {
NavigationStack {
ConversationScreen()
}
}
}
Loading
Loading