Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Vertex AI] Add ImageGenerationInstance for input to predict call #14202

Merged
merged 1 commit into from
Dec 3, 2024

Conversation

andrewheard
Copy link
Contributor

Added an encodable type ImageGenerationInstance that is passed in instances[] in a predict request. The schema (download) supports other fields, such as image and mask, but we are only using prompt at this time.

VisionGenerativeModel Instance Schema Reference
title: VisionGenerativeModel
type: object
required:
- prompt
properties:
  prompt:
    type: string
    description: >
      The text prompt for guiding how vision model generate the images.
      This field is required for both generation and editing.
  image:
    type: object
    description: >
      Image for editing. This field is required for editing. Not needed for generation.
    oneof:
    - bytesBase64Encoded
    - gcsUri
    properties:
      bytesBase64Encoded:
        type: string
        description: Image bytes encoded in base64 string.
      gcsUri:
        type: string
        description: >
          The Google Cloud Storage location of the image on which to perform the editing.
        pattern: '^gs:\/\/(.+)\/(.+)$'
      mimeType:
        type: string
        description: >
          The MIME type of the content of the image. Only the images in below listed
          MIME types are supported.
        enum:
        - image/jpeg
        - image/png
  mask:
    type: object
    description: >
      Mask where to edit. This is an optional field for editing. No need for generation.
    oneof:
    - image
    - polygonList
    properties:
      polygonList:
        type: array
        description: Multiple polygon masks.
        items:
          type: array
          description: >
            All of the vertices that form the single polygon mask.
          items:
            type: object
            description: >
              Single vertex with `x` and `y` coordinates that describes a point in 2D plane.
            properties:
              x:
                type: number
                format: float
                minimum: 0.0
                maximum: 1.0
                description: The x coordinate of the vertex.
              y:
                type: number
                format: float
                minimum: 0.0
                maximum: 1.0
                description: The y coordinate of the vertex.

#no-changelog

@andrewheard andrewheard marked this pull request as ready for review December 3, 2024 03:51
@andrewheard andrewheard requested a review from paulb777 December 3, 2024 03:57
@andrewheard andrewheard merged commit 37e2390 into vertex-imagen Dec 3, 2024
46 checks passed
@andrewheard andrewheard deleted the ah/vertex-imagen-instance branch December 3, 2024 04:20
@firebase firebase locked and limited conversation to collaborators Jan 3, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants