ColStr.Append has performance problem

[hongker]

insert 100k records by Batch.Append function, I found it's has performance problem, current code is:

// code position: proto/col_str.go(23)
// Append string to column.
func (c *ColStr) Append(v string) {
	start := len(c.Buf)
	c.Buf = append(c.Buf, v...)
	end := len(c.Buf)
	c.Pos = append(c.Pos, Position{Start: start, End: end})
}

the cap of c.Buf not pre define, so it will took much time on runtime.growslice when insert too many records,

// pprof result: 
----------------------------------------------------------+-------------
                                            31.40s   100% |   github.com/ClickHouse/clickhouse-go/v2/lib/column.(*String).AppendRow
     2.34s  0.72%  5.27%     31.40s  9.72%                | github.com/ClickHouse/ch-go/proto.(*ColStr).Append
                                            22.94s 73.06% |   runtime.growslice
                                             5.82s 18.54% |   runtime.memmove
                                             0.27s  0.86% |   runtime.gcWriteBarrierR8
                                             0.02s 0.064% |   runtime.asyncPreempt
                                             0.01s 0.032% |   runtime.gcWriteBarrier
----------------------------------------------------------+-------------

Describe the solution you'd like
I want a function, it can pre define the cap of col.Buf, like this:

func (c *ColStr) WithBufCap(cap int) {
        c.Buf = make([]byte, 0, cap)
}

[jkaflik]

Hi @hongker
would you mind submitting a proposal PR?

[kevinmingtarja]

Hi, I'd like to contribute a PR if no-one is working on this.

[jkaflik]

@kevinmingtarja go ahead. Contributions are appreciated.

[hongker]

Hi, I'd like to contribute a PR if no-one is working on this.

did you finished this work?

[kevinmingtarja]

Hi @hongker, no I just started. But if you want to do it, please go ahead since you are the one who proposed the feature.