Maybe not the right tool for my need

[m4nch0t]

Hi,

I need to build a custom OMI for Outscale. I don't want to start with the centos OMI present in the marketplace because it doesn't satisfy my requirements for partitioning. So in the past,I've already work with packer and vsphere-iso to boot from an iso or an url and download a kickstart for installing my RHEL/CentOS. But I don't see how to achieve the same things with osc-bsusurrogate or osc-chroot. I miss something? Maybe I can achieve this with an other method?

Thank you for your help.

My surrogate_centos-8-stream.json.pkr.hcl :

source "osc-bsusurrogate" "autogenerated_1" {
access_key = "my_ak"
associate_public_ip_address = true
launch_block_device_mappings {
delete_on_vm_deletion = false
device_name = "/dev/xvdf"
iops = 300
volume_size = 10
volume_type = "io1"
}
omi_block_device_mappings {
device_name = "/dev/sda2"
volume_size = 10
volume_type = "standard"
}
omi_description = "custom centos"
omi_name = "packer-msi"
omi_root_device {
delete_on_vm_deletion = true
device_name = "/dev/sda1"
source_device_name = "/dev/xvdf"
volume_size = 10
volume_type = "standard"
}
omi_virtualization_type = "hvm"
region = "eu-west-2"
secret_key = "my_secretkey"
source_omi = "ami-0dd0ab23"
ssh_username = "outscale"
subnet_id = "subnet-id"
subregion_name = "eu-west-2a"
vm_type = "t2.medium"
}
build {
sources = ["source.osc-bsusurrogate.autogenerated_1"]
}

My config.pkr.hcl :

packer {
required_plugins {
outscale = {
version = ">= 1.0.0"
source = "github.com/hashicorp/outscale"
}
}
}

My packer command :

docker run --rm\ --platform linux/amd64 \ -vpwd:/workspace -w /workspace \ -e PACKER_PLUGIN_PATH=/workspace/.packer.d/plugins \ hashicorp/packer:latest \ init config.pkr.hcl && packer build .

The output command (same with -debug ) :

`
osc-bsusurrogate.autogenerated_1: output will be in this color.

==> osc-bsusurrogate.autogenerated_1: Prevalidating OMI Name: packer-msi
osc-bsusurrogate.autogenerated_1: Found Image ID: ami-0dd0ab23
==> osc-bsusurrogate.autogenerated_1: Creating temporary keypair: packer_615d5585-bcee-1963-0880-68888745700b
==> osc-bsusurrogate.autogenerated_1: Creating temporary PublicIp for instance
==> osc-bsusurrogate.autogenerated_1: Creating temporary security group for this instance: packer_osc_615d5586-0f48-8c2d-a421-a194c324af49
==> osc-bsusurrogate.autogenerated_1: Authorizing access to port 22 from 0.0.0.0/0 in the temporary security group...
==> osc-bsusurrogate.autogenerated_1: Launching a source OUTSCALE vm...
==> osc-bsusurrogate.autogenerated_1: Adding tags to source vm
osc-bsusurrogate.autogenerated_1: Vm ID: i-5624caa7
==> osc-bsusurrogate.autogenerated_1: Waiting for vm (i-5624caa7) to become ready...
osc-bsusurrogate.autogenerated_1: Adding tag: "Name": "Packer Builder"
==> osc-bsusurrogate.autogenerated_1: Linking temporary PublicIp eipalloc-908b0ae4 to instance i-5624caa7
==> osc-bsusurrogate.autogenerated_1: Using SSH communicator to connect: 80.247.0.99
==> osc-bsusurrogate.autogenerated_1: Waiting for SSH to become available...
==> osc-bsusurrogate.autogenerated_1: Connected to SSH!
==> osc-bsusurrogate.autogenerated_1: Stopping the source vm...
osc-bsusurrogate.autogenerated_1: Stopping vm, attempt 1
==> osc-bsusurrogate.autogenerated_1: Waiting for the vm to stop...
==> osc-bsusurrogate.autogenerated_1: Creating snapshot of EBS Volume vol-fa17b08f...
==> osc-bsusurrogate.autogenerated_1: Registering the OMI...
==> osc-bsusurrogate.autogenerated_1: Error registering OMI: 400 Bad Request
==> osc-bsusurrogate.autogenerated_1: Removing snapshots since we cancelled or halted...
==> osc-bsusurrogate.autogenerated_1: Provisioning step had errors: Running the cleanup provisioner, if present...
==> osc-bsusurrogate.autogenerated_1: Terminating the source OUTSCALE vm...
==> osc-bsusurrogate.autogenerated_1: Cleaning up any extra volumes...
==> osc-bsusurrogate.autogenerated_1: Deleting temporary security group...
==> osc-bsusurrogate.autogenerated_1: Deleting temporary PublicIp...
==> osc-bsusurrogate.autogenerated_1: Deleting temporary keypair...
Build 'osc-bsusurrogate.autogenerated_1' errored after 2 minutes 28 seconds: Error registering OMI: 400 Bad Request

==> Wait completed after 2 minutes 28 seconds

==> Some builds didn't complete successfully and had errors:
--> osc-bsusurrogate.autogenerated_1: Error registering OMI: 400 Bad Request

==> Builds finished but no artifacts were created.
`

[outscale-mdr]

Hello @m4nch0t,

If you want to create an image from scratch, checkout our examples in this repository (https://github.com/outscale/omi-packer).

On the other side, we are currently investigating your example and we will update you on the subject.

Best regards,
Maxime

[outscale-mdr]

Update 1:

1. omi_block_device_mappings is usually used with chroot builder to add volumes at runtime but in your case you have to use multiple launch_block_device_mappingsto add volumes at the boot of the created VM (we'll look on how to avoid this)
2. The name of the device shoud be /dev/sdX or /dev/xvfX and not /dev/sdX1 because it must be a device name not a partition name (we'll look on that to improve the parsing)

[m4nch0t]

Thank you for your response,
I fix the Update 1 / 2. by replacing device_name": "/dev/sda1" with device_name": "/dev/sda".
I also attach the debug.log generated with PACKER_LOG=1 var.
debug.log

For omi-packer, I've already see it, but don't even try. You use the official qcow2 from CentOS, but I don't want to use it because I don't want this partition scheme (not compliant with CIS hardening guide), and optionally this filesystem (the goal is to use lvm/xfs). Maybe I mistranslate the documentation...

[outscale-mdr]

Here is your hcl fixed:

source "osc-bsusurrogate" "autogenerated_1" {
  associate_public_ip_address = true
  launch_block_device_mappings {
    delete_on_vm_deletion = false
    device_name           = "/dev/xvdf"
    iops                  = 300
    volume_size           = 10
    volume_type           = "io1"
  }
  omi_block_device_mappings {
    device_name = "/dev/xvde"
    volume_size = 10
    volume_type = "standard"
  }
  omi_description = "custom centos"
  omi_name        = "packer-msi"
  omi_root_device {
    delete_on_vm_deletion = true
    device_name           = "/dev/sda1"
    source_device_name    = "/dev/xvdf"
    volume_size           = 10
    volume_type           = "standard"
  }
  omi_virtualization_type = "hvm"
  region                  = "eu-west-2"
  source_omi              = "ami-0dd0ab23"
  ssh_username            = "outscale"
  subregion_name          = "eu-west-2a"
  vm_type                 = "t2.medium"
}
build {
  sources = ["source.osc-bsusurrogate.autogenerated_1"]
}

For your use-case, the chroot builder could be useful because you can create from scratch (with the option from_scratch) your volume. However currently, it will take you a lot of time to install a CentOS from scratch because we do not support booting from ISO.

[m4nch0t]

Thank you, so I've to remove :

• security_group_id
• subnet_id

Change

• omi_block_device_mappings. device_name="/dev/xvdf "with /dev/xvde
• omi_root_device. device_name="/dev/sda" with "/dev/sda1"

With this, I've now :
osc-bsusurrogate.autogenerated_1: Error registering OMI: 500 Internal Server Error

If I keep omi_root_device. device_name="/dev/sda" I still have
osc-bsusurrogate.autogenerated_1: Error registering OMI: 400 Bad Request

Nevermind, if this builder is not compliant with my need, don't need to dig further. I'll take another look at osc-chroot

[m4nch0t]

Hello again,

I'm trying to use osc-chroot, with the exact same "basic example", got some typos in the doc.

An argument named "ami_name" is not expected here. Did you mean "omi_name"?

Error: Unsupported argument

on scratch_centos-8-stream.json.pkr.hcl line 12:
2021/10/07 11:10:21 [INFO] (telemetry) Finalizing.
(source code not available)

An argument named "ami_virtualization_type" is not expected here. Did you mean
"omi_virtualization_type"?

Error: Unsupported block type

on scratch_centos-8-stream.json.pkr.hcl line 6:
(source code not available)

Blocks of type "ami_block_device_mappings" are not expected here. Did you mean
"omi_block_device_mappings"?

When I change ami/omi, got :

==> Builds finished but no artifacts were created.
An argument named "delete_on_termination" is not expected here.

If I remove it, got the same error with my first custom config :

==> osc-chroot.autogenerated_1: Gathering information about this Outscale vm...
==> osc-chroot.autogenerated_1: Please verify Packer is running on a proper Outscale vm.
==> osc-chroot.autogenerated_1: Error retrieving the ID of the vm Packer is running on.
==> osc-chroot.autogenerated_1: Please verify Packer is running on a proper Outscale vm.
2021/10/07 11:14:51 [INFO] (telemetry) ending osc-chroot.autogenerated_1
Please verify Packer is running on a proper Outscale vm.
Build 'osc-chroot.autogenerated_1' errored after 318 milliseconds 482 microseconds: Error retrieving the ID of the vm Packer is running on.
Please verify Packer is running on a proper Outscale vm.

I've launched a VM (named packer) with packer installed, but maybe I'm not fully awake and miss something in the doc, but I don't see any reference in the documentation of this builder...

Best regards

[outscale-mdr]

You are right, thank your for this. We will update the doc.

Just to be sure, you are executing packer from a VM from the Outscale cloud ?

[m4nch0t]

Oh, I mistranslate the doc, I have to run from a packer vm in my vpc, not the local packer who connect to my vm with my key pair. I'll give it a try.
That's explain why my filter to find the packer vm is useless and don't work.

[m4nch0t]

Ok, I'm now in a fresh install of ami-0dd0ab23 in my vpc, I've installed packer 1.7.6.
When I try with outscale user the command packer build . , I've an error on permissions :

==> osc-chroot.autogenerated_1: Error creating lock: mkdir /var/lock/packer-chroot: permission denied

If I re run the same command with sudo (or directly with root user) nothing happen, any traffic captured with tcpdump or output in console , even with PACKER_LOG=1. Same behaviour with packer init.

[m4nch0t]

Hello,
We can close the issue, we code an a single role to achieve this, easier way to achieve our goals.
Thanks