perf(errors/gerror): 缓存堆栈信息 #4071

wln32 · 2024-12-21T14:47:16Z

为gerror缓存堆栈信息，缓存之后，性能提升几倍，内存申请也减少了许多，不过目前仍有一些问题与主分支的代码输出的结果不同，
gerror类型使用Wrap再次包装之后，再次输出的时候，两个错误的堆栈信息合并到一起了，这样看起来能够大幅度减少一些噪音，但似乎Wrap之后的error堆栈信息丢失了，只能看到一个“gerror/Wrap包装数据3”错误信息，这样是否不妥？
我使用"github.com/pkg/errors"这个包，使用相同的逻辑，使用errors.New，然后在Test3函数重新使用errors.Wrap包装一下，再次返回，输出的结果也是两次堆栈信息都是独立输出的，并没有合并到一起去

此外本次提交的代码没有忽略标准库的路径还有编译器自动生成的代码路径

func benchGfErrorsTest1() error {
	return benchGfErrorsTest2()
}

func benchGfErrorsTest2() error {
	return benchGfErrorsTest3()
}

func benchGfErrorsTest3() error {
	return gerror.Wrap(benchGfErrorsTest4(), "gerror/Wrap包装数据3")
}

func benchGfErrorsTest4() error {
	return benchGfErrorsTest5()
}

func benchGfErrorsTest5() error {
	return gerror.New("gerror/New测试数据")
}

func main() {
	err := benchGfErrorsTest1()
	fmt.Printf("%+v\n", err)
}

运行上面代码

gf的输出结果为

gerror/Wrap包装数据3: gerror/New测试数据
1. gerror/Wrap包装数据3
2. gerror/New测试数据
   1).  main.benchGfErrorsTest5
        go-test/test.go:63
   2).  main.benchGfErrorsTest4
        go-test/test.go:59
   3).  main.benchGfErrorsTest3
        go-test/test.go:55
   4).  main.benchGfErrorsTest2
        go-test/test.go:51
   5).  main.benchGfErrorsTest1
        go-test/test.go:47
   6).  main.main
        go-test/gerror.go:6

此次pr的输出结果

gerror/Wrap包装数据3: gerror/New测试数据
1. gerror/Wrap包装数据3
        1). main.benchGfErrorsTest3
                go-test/test.go:55
        2). main.benchGfErrorsTest2
                go-test/test.go:51
        3). main.benchGfErrorsTest1
                go-test/test.go:47
        4). main.main
                go-test/gerror.go:6
        5). runtime.main
                I:/go1.22/src/runtime/proc.go:271
        6). runtime.goexit
                I:/go1.22/src/runtime/asm_amd64.s:1695
2. gerror/New测试数据
        1). main.benchGfErrorsTest5
                go-test/test.go:63
        2). main.benchGfErrorsTest4
                go-test/test.go:59
        3). main.benchGfErrorsTest3
                go-test/test.go:55
        4). main.benchGfErrorsTest2
                go-test/test.go:51
        5). main.benchGfErrorsTest1
                go-test/test.go:47
        6). main.main
                go-test/gerror.go:6
        7). runtime.main
                I:/go1.22/src/runtime/proc.go:271
        8). runtime.goexit
                I:/go1.22/src/runtime/asm_amd64.s:1695

性能比对

分为直接用new返回，中间不做任何处理，还有一个是中间用Wrap包装了下new返回的error
都使用%+v解开堆栈信息来测试

// 测试命令 go test -v -run=none -bench=^Benchmark_gfErrors -benchtime=5s -benchmem -cpuprofile='cpu.out' -memprofile='mem.out' -count=3 .
func Benchmark_gfErrors(b *testing.B) {
	b.ReportAllocs()
	var err error
	s := " "
	for range b.N {
		err = benchGfErrorsTest1()
		s = fmt.Sprintf("%+v", err)
	}
	_ = s
}

func Benchmark_gfErrors_Parallel(b *testing.B) {
	b.ReportAllocs()
	var err error
	s := " "
	b.RunParallel(func(p *testing.PB) {
		for p.Next() {
			err = benchGfErrorsTest1()
			s = fmt.Sprintf("%+v", err)
		}
	})
	_ = s
}

当前主分支的new

Benchmark_gfErrors
Benchmark_gfErrors-8            	  320312	     15957 ns/op	    8040 B/op	      70 allocs/op
Benchmark_gfErrors-8            	  359022	     15981 ns/op	    8040 B/op	      70 allocs/op
Benchmark_gfErrors-8            	  361665	     15617 ns/op	    8040 B/op	      70 allocs/op
Benchmark_gfErrors_Parallel
Benchmark_gfErrors_Parallel-8   	 1000000	      5350 ns/op	    8185 B/op	      70 allocs/op
Benchmark_gfErrors_Parallel-8   	 1000000	      5108 ns/op	    8184 B/op	      70 allocs/op
Benchmark_gfErrors_Parallel-8   	 1000000	      5140 ns/op	    8183 B/op	      70 allocs/op

当前主分支的wrap

Benchmark_gfErrors
Benchmark_gfErrors-8            	  256899	     24295 ns/op	    8954 B/op	     101 allocs/op
Benchmark_gfErrors-8            	  249151	     24198 ns/op	    8954 B/op	     101 allocs/op
Benchmark_gfErrors-8            	  264451	     23054 ns/op	    8953 B/op	     101 allocs/op
Benchmark_gfErrors_Parallel
Benchmark_gfErrors_Parallel-8   	  783068	      7812 ns/op	   10508 B/op	     102 allocs/op
Benchmark_gfErrors_Parallel-8   	  694832	      7815 ns/op	   10506 B/op	     102 allocs/op
Benchmark_gfErrors_Parallel-8   	  872317	      7450 ns/op	   10506 B/op	     102 allocs/op

当前pr的new

Benchmark_gfErrors
Benchmark_gfErrors-8            	 1912519	      3117 ns/op	    1794 B/op	       6 allocs/op
Benchmark_gfErrors-8            	 1857090	      3092 ns/op	    1794 B/op	       6 allocs/op
Benchmark_gfErrors-8            	 1974368	      3112 ns/op	    1794 B/op	       6 allocs/op
Benchmark_gfErrors_Parallel
Benchmark_gfErrors_Parallel-8   	 6405152	      1009 ns/op	    1761 B/op	       6 allocs/op
Benchmark_gfErrors_Parallel-8   	 6406886	       966.3 ns/op	    1761 B/op	       6 allocs/op
Benchmark_gfErrors_Parallel-8   	 5790344	       951.7 ns/op	    1761 B/op	       6 allocs/op

当前pr的wrap

Benchmark_gfErrors
Benchmark_gfErrors-8            	 1000000	      5614 ns/op	    3366 B/op	      13 allocs/op
Benchmark_gfErrors-8            	 1000000	      5713 ns/op	    3366 B/op	      13 allocs/op
Benchmark_gfErrors-8            	 1000000	      5631 ns/op	    3366 B/op	      13 allocs/op
Benchmark_gfErrors_Parallel
Benchmark_gfErrors_Parallel-8   	 3922666	      1579 ns/op	    3172 B/op	      13 allocs/op
Benchmark_gfErrors_Parallel-8   	 3632580	      1635 ns/op	    3172 B/op	      13 allocs/op
Benchmark_gfErrors_Parallel-8   	 3618468	      1620 ns/op	    3172 B/op	      13 allocs/op

wln32 · 2024-12-21T14:47:48Z

errors/gerror/gerror_z_unit_stack_test.go

+// You can obtain one at https://github.com/gogf/gf.
+
+package gerror_test
+


这个文件稍后会删掉

gqcn · 2024-12-23T01:43:25Z

@wln32

gerror类型使用Wrap再次包装之后，再次输出的时候，两个错误的堆栈信息合并到一起了，这样看起来能够大幅度减少一些噪音，但似乎Wrap之后的error堆栈信息丢失了，只能看到一个“gerror/Wrap包装数据3”错误信息，这样是否不妥？

这块是goframe的错误处理组件特意做的降噪处理，因为Wrap后的error有重复的代码行，特别是针对多次Warp后层级比较深的error，效果比较明显，也不影响问题定位。

关于gerror的性能改进。

由于error只在错误产生时才会执行调用，虽然生成堆栈的性能会较差，但是也不是什么问题。缓存这版实现我没有看明白，并且gerror是核心组件，建议类似改动可以先提交draft描述思路设置设计方案，再去落地代码实现。

Issues-translate-bot · 2024-12-23T01:43:37Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@wln32

After the gerror type is wrapped again using Wrap, when it is output again, the stack information of the two errors is merged together. This seems to be able to greatly reduce some noise, but it seems that the error stack information after Wrap is lost and can only be seen. A "gerror/Wrap packaged data 3" error message, is this inappropriate?

This is the noise reduction process specially done by the error handling component of goframe, because the error after Wrap has repeated lines of code, especially for the deeper error after multiple Warp, the effect is It is more obvious and does not affect the problem location.

Performance improvements on gerror.

Since error is only called when an error occurs, although the performance of generating the stack will be poor, it is not a problem. I don’t understand the implementation of this version of caching, and gerror is a core component. It is recommended that for similar changes, you can first submit a draft to describe the idea and set up the design plan, and then implement the code.

wln32 · 2024-12-23T11:45:54Z

@wln32

gerror类型使用Wrap再次包装之后，再次输出的时候，两个错误的堆栈信息合并到一起了，这样看起来能够大幅度减少一些噪音，但似乎Wrap之后的error堆栈信息丢失了，只能看到一个“gerror/Wrap包装数据3”错误信息，这样是否不妥？

这块是goframe的错误处理组件特意做的降噪处理，因为Wrap后的error有重复的代码行，特别是针对多次Warp后层级比较深的error，效果比较明显，也不影响问题定位。

关于gerror的性能改进。

由于error只在错误产生时才会执行调用，虽然生成堆栈的性能会较差，但是也不是什么问题。缓存这版实现我没有看明白，并且gerror是核心组件，建议类似改动可以先提交draft描述思路设置设计方案，再去落地代码实现。

@gqcn

这块是goframe的错误处理组件特意做的降噪处理，因为Wrap后的error有重复的代码行，特别是针对多次Warp后层级比较深的error，效果比较明显，也不影响问题定位。

这样还是有影响的吧，只能看到是两个错误，然后是堆栈信息，但是并不知道哪个错误的堆栈在哪里，还得一个个看

由于error只在错误产生时才会执行调用，虽然生成堆栈的性能会较差，但是也不是什么问题

不能认同

缓存这版实现我没有看明白

大概思路就是使用先前保存的pc指针，也就是每一层调用栈函数的pc的指针，

然后调用runtime.CallersFrames(pcs[:]).Next()来获取函数的名字，行号，文件名等信息，
这一步会做缓存，先根据pc指针在缓存中查找，如果找到了，就不用在调用runtime库重复获取，这个api调用比较耗时。
如果没找到，再调用api将其添加到缓存中
使用bytes.Buffer或者strings.Builder这样的带缓冲的结构来保存本次需要打印的信息，如果在调用Write时发生了扩容，这也会比较耗时，所以这部分的思路就是用一个count变量统计本次需要打印的堆栈信息的长度，直接一次扩容，确保不会再写入时再次发生扩容，直接写入即可
然后就是把每次使用的bytes.Buffer放进sync.Pool来做缓存，也可以减少gc或者内存申请的次数

建议类似改动可以先提交draft描述思路设置设计方案，再去落地代码实现。

其实这次pr的想法是我看到了一篇文章，里面介绍了可以把堆栈的一些信息给保存起来，再次使用的时候，不需要再次调用runtime库获取，我在本地实现了一下，代码其实也不多，就四五十行左右（本地那版实现比现在的性能还要高出两倍），我就想尝试在gerror看能不能实现下，目前来说，可以实现的和先前的效果一模一样，没什么难度
这种思路其实还可以用在其他地方，像glog，需要打印文件名函数名行号的，也都可以做到获取一次，后续重复使用，几乎没有什么成本

Issues-translate-bot · 2024-12-23T11:46:04Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@wln32

After the gerror type is wrapped again using Wrap, when it is output again, the stack information of the two errors is merged together. This seems to be able to greatly reduce some noise, but it seems that the error stack information after Wrap is lost and can only be viewed. I get a "gerror/Wrap package data 3" error message. Is this inappropriate?

This is a noise reduction process specially performed by the error handling component of goframe, because the error after Wrap has repeated lines of code, especially for the deeper error after multiple Warp. The effect is obvious and does not affect problem location.

Performance improvements on gerror.

Since error is called only when an error occurs, although the performance of generating the stack will be poor, it is not a problem. I don’t understand the implementation of this version of caching, and gerror is a core component. It is recommended that for similar changes, you can first submit a draft to describe the idea and set up the design plan, and then implement the code.

@gqcn

This is a noise reduction process specially done by the error handling component of goframe, because the error after Wrap has repeated lines of code, especially for the error with a deeper level after multiple Warp , the effect is relatively obvious and does not affect problem location.

This still has an impact. I can only see two errors and then the stack information, but I don’t know where the stack of which error is. I have to look at it one by one.

Since error will only be called when an error occurs, although the performance of generating the stack will be poor, it is not a problem.

Can't agree

I don’t understand the implementation of this version of caching.

The general idea is to use the previously saved pc pointer, which is the pc pointer of the call stack function at each layer.

Then call runtime.CallersFrames(pcs[:]).Next() to obtain the function name, line number, file name and other information,
This step will be cached. First, search the cache according to the pc pointer. If it is found, there is no need to call the runtime library to obtain it repeatedly. This API call is time-consuming.
If not found, call the API to add it to the cache.
Use a buffered structure such as bytes.Buffer or strings.Builder to save the information that needs to be printed this time. If expansion occurs when calling Write, this will also be more time-consuming, so the idea in this part is to use a count The variable counts the length of the stack information that needs to be printed this time. It can be directly expanded once to ensure that it will not be expanded again when writing. Just write directly.
Then put the bytes.Buffer used each time into sync.Pool for caching, which can also reduce the number of gc or memory applications.

It is recommended that for similar changes, you can first submit a draft to describe the idea and set up the design plan, and then implement the code.

In fact, the idea for this PR was that I saw an article that introduced how to save some information on the stack. When using it again, there is no need to call the runtime library again to obtain it. I implemented it locally, and the code is actually the same. Not much, just about forty or fifty lines (the local version is twice as fast as the current one), so I want to try it in gerror to see if it can be implemented. At present, the effect that can be achieved is exactly the same as the previous one. , no difficulty
This idea can actually be used in other places, such as glog. If you need to print the file name, function name and line number, you can also obtain it once and reuse it later, with almost no cost.

houseme · 2024-12-24T10:46:49Z

@wln32

gerror类型使用Wrap再次包装之后，再次输出的时候，两个错误的堆栈信息合并到一起了，这样看起来能够大幅度减少一些噪音，但似乎Wrap之后的error堆栈信息丢失了，只能看到一个“gerror/Wrap包装数据3”错误信息，这样是否不妥？

这块是goframe的错误处理组件特意做的降噪处理，因为Wrap后的error有重复的代码行，特别是针对多次Warp后层级比较深的error，效果比较明显，也不影响问题定位。

关于gerror的性能改进。

由于error只在错误产生时才会执行调用，虽然生成堆栈的性能会较差，但是也不是什么问题。缓存这版实现我没有看明白，并且gerror是核心组件，建议类似改动可以先提交draft描述思路设置设计方案，再去落地代码实现。

@gqcn

这块是goframe的错误处理组件特意做的降噪处理，因为Wrap后的error有重复的代码行，特别是针对多次Warp后层级比较深的error，效果比较明显，也不影响问题定位。

这样还是有影响的吧，只能看到是两个错误，然后是堆栈信息，但是并不知道哪个错误的堆栈在哪里，还得一个个看

由于error只在错误产生时才会执行调用，虽然生成堆栈的性能会较差，但是也不是什么问题

不能认同

缓存这版实现我没有看明白

大概思路就是使用先前保存的pc指针，也就是每一层调用栈函数的pc的指针，

然后调用runtime.CallersFrames(pcs[:]).Next()来获取函数的名字，行号，文件名等信息，
这一步会做缓存，先根据pc指针在缓存中查找，如果找到了，就不用在调用runtime库重复获取，这个api调用比较耗时。
如果没找到，再调用api将其添加到缓存中

使用bytes.Buffer或者strings.Builder这样的带缓冲的结构来保存本次需要打印的信息，如果在调用Write时发生了扩容，这也会比较耗时，所以这部分的思路就是用一个count变量统计本次需要打印的堆栈信息的长度，直接一次扩容，确保不会再写入时再次发生扩容，直接写入即可

然后就是把每次使用的bytes.Buffer放进sync.Pool来做缓存，也可以减少gc或者内存申请的次数

建议类似改动可以先提交draft描述思路设置设计方案，再去落地代码实现。

其实这次pr的想法是我看到了一篇文章，里面介绍了可以把堆栈的一些信息给保存起来，再次使用的时候，不需要再次调用runtime库获取，我在本地实现了一下，代码其实也不多，就四五十行左右（本地那版实现比现在的性能还要高出两倍），我就想尝试在gerror看能不能实现下，目前来说，可以实现的和先前的效果一模一样，没什么难度这种思路其实还可以用在其他地方，像glog，需要打印文件名函数名行号的，也都可以做到获取一次，后续重复使用，几乎没有什么成本

建议搞一个 glog 的改造，这个使用的地方更多

Issues-translate-bot · 2024-12-24T10:47:00Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@wln32

After the gerror type is wrapped again using Wrap, when it is output again, the stack information of the two errors is merged together. This seems to be able to greatly reduce some noise, but it seems that the error stack information after Wrap is lost. I see a "gerror/Wrap package data 3" error message. Is this inappropriate?

This is a noise reduction process specially done by the error handling component of goframe, because the error after Wrap has repeated lines of code, especially for the error with a deeper level after multiple Warp , the effect is relatively obvious and does not affect problem location.

Performance improvements on gerror.

Since error will only be called when an error occurs, although the performance of generating the stack will be poor, it is not a problem. I don’t understand the implementation of this version of caching, and gerror is a core component. It is recommended that for similar changes, you can first submit a draft to describe the idea and set up the design plan, and then implement the code.

@gqcn

This is a noise reduction process specially done by the error handling component of goframe, because the error after Wrap has repeated lines of code, especially for the error with a deeper level after multiple Warp , the effect is more obvious and does not affect the problem location.

This still has an impact. I can only see two errors and then the stack information, but I don’t know where the stack of which error is. I have to look at it one by one.

Since error is only called when an error occurs, although the performance of generating the stack will be poor, it is not a problem.

Cannot agree

I don’t understand this version of cache implementation.

The general idea is to use the previously saved pc pointer, which is the pc pointer of the call stack function at each layer.

Then call runtime.CallersFrames(pcs[:]).Next() to obtain the function name, line number, file name and other information,
This step will be cached. First, search the cache according to the pc pointer. If it is found, there is no need to call the runtime library to obtain it repeatedly. This API call is time-consuming.
If not found, call the api to add it to the cache

Use a buffered structure such as bytes.Buffer or strings.Builder to save the information that needs to be printed this time. If expansion occurs when calling Write, this will also be more time-consuming, so the idea in this part is to use a The count variable counts the length of the stack information that needs to be printed this time. It can be directly expanded once to ensure that it will not be expanded again when writing. Just write directly.

Then put the bytes.Buffer used each time into sync.Pool for caching, which can also reduce the number of gc or memory applications.

It is recommended that for similar changes, you can first submit a draft to describe the idea and set up the design plan, and then implement the code.

In fact, the idea behind this PR was that I saw an article that introduced how to save some information of the stack. When using it again, there is no need to call the runtime library again to obtain it. I implemented it locally. The code is actually It’s not much, only about forty or fifty lines (the local version is twice as fast as the current performance), so I want to try gerror to see if it can be implemented. At present, what can be achieved is the same as the previous effect. Exactly the same, no difficulty This idea can actually be used in other places, such as glog. If you need to print the file name, function name and line number, you can also obtain it once and reuse it later, with almost no cost.

It is recommended to make a transformation of glog, which can be used in more places.

wln32 · 2024-12-24T13:04:53Z

@wln32

gerror类型使用Wrap再次包装之后，再次输出的时候，两个错误的堆栈信息合并到一起了，这样看起来能够大幅度减少一些噪音，但似乎Wrap之后的error堆栈信息丢失了，只能看到一个“gerror/Wrap包装数据3”错误信息，这样是否不妥？

这块是goframe的错误处理组件特意做的降噪处理，因为Wrap后的error有重复的代码行，特别是针对多次Warp后层级比较深的error，效果比较明显，也不影响问题定位。

关于gerror的性能改进。

由于error只在错误产生时才会执行调用，虽然生成堆栈的性能会较差，但是也不是什么问题。缓存这版实现我没有看明白，并且gerror是核心组件，建议类似改动可以先提交draft描述思路设置设计方案，再去落地代码实现。

@gqcn

这块是goframe的错误处理组件特意做的降噪处理，因为Wrap后的error有重复的代码行，特别是针对多次Warp后层级比较深的error，效果比较明显，也不影响问题定位。

这样还是有影响的吧，只能看到是两个错误，然后是堆栈信息，但是并不知道哪个错误的堆栈在哪里，还得一个个看

由于error只在错误产生时才会执行调用，虽然生成堆栈的性能会较差，但是也不是什么问题

不能认同

缓存这版实现我没有看明白

大概思路就是使用先前保存的pc指针，也就是每一层调用栈函数的pc的指针，

然后调用runtime.CallersFrames(pcs[:]).Next()来获取函数的名字，行号，文件名等信息，
这一步会做缓存，先根据pc指针在缓存中查找，如果找到了，就不用在调用runtime库重复获取，这个api调用比较耗时。
如果没找到，再调用api将其添加到缓存中

使用bytes.Buffer或者strings.Builder这样的带缓冲的结构来保存本次需要打印的信息，如果在调用Write时发生了扩容，这也会比较耗时，所以这部分的思路就是用一个count变量统计本次需要打印的堆栈信息的长度，直接一次扩容，确保不会再写入时再次发生扩容，直接写入即可

然后就是把每次使用的bytes.Buffer放进sync.Pool来做缓存，也可以减少gc或者内存申请的次数

建议类似改动可以先提交draft描述思路设置设计方案，再去落地代码实现。

其实这次pr的想法是我看到了一篇文章，里面介绍了可以把堆栈的一些信息给保存起来，再次使用的时候，不需要再次调用runtime库获取，我在本地实现了一下，代码其实也不多，就四五十行左右（本地那版实现比现在的性能还要高出两倍），我就想尝试在gerror看能不能实现下，目前来说，可以实现的和先前的效果一模一样，没什么难度这种思路其实还可以用在其他地方，像glog，需要打印文件名函数名行号的，也都可以做到获取一次，后续重复使用，几乎没有什么成本

建议搞一个 glog 的改造，这个使用的地方更多

好想法

Issues-translate-bot · 2024-12-24T13:05:06Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@wln32

After the gerror type is wrapped again using Wrap, when it is output again, the stack information of the two errors is merged together. This seems to be able to greatly reduce some noise, but it seems that the error stack information after Wrap is lost. I can see a "gerror/Wrap package data 3" error message. Is this inappropriate?

This is a noise reduction process specially done by the error handling component of goframe, because the error after Wrap has repeated lines of code, especially for the error with a deeper level after multiple Warp , the effect is more obvious and does not affect the problem location.

Performance improvements on gerror.

Since error is only called when an error occurs, although the performance of generating the stack will be poor, it is not a problem. I don’t understand the implementation of this version of caching, and gerror is a core component. It is recommended that for similar changes, you can first submit a draft to describe the idea and set up the design plan, and then implement the code.

@gqcn

This is a noise reduction process specially done by the error handling component of goframe, because the error after Wrap has repeated lines of code, especially for those with deeper levels after multiple Warp error`, the effect is more obvious and does not affect problem location.

This still has an impact. I can only see two errors and then the stack information, but I don’t know where the stack of which error is. I have to look at it one by one.

Since error is only called when an error occurs, although the performance of generating the stack will be poor, it is not a problem.

Cannot agree

I don’t understand the cache implementation.

The general idea is to use the previously saved pc pointer, which is the pointer of the pc for each layer of call stack functions.

Then call runtime.CallersFrames(pcs[:]).Next() to obtain the function name, line number, file name and other information,
This step will do caching. First, search the cache according to the pc pointer. If it is found, there is no need to call the runtime library to obtain it repeatedly. This API call is time-consuming.
If not found, call the api to add it to the cache

Use a buffered structure such as bytes.Buffer or strings.Builder to save the information that needs to be printed this time. If expansion occurs when calling Write, this will also be more time-consuming, so the idea in this part is to use A count variable counts the length of the stack information that needs to be printed this time. It can be directly expanded once to ensure that it will not be expanded again when writing. Just write directly.

Then put the bytes.Buffer used each time into sync.Pool for caching, which can also reduce the number of gc or memory applications.

It is recommended that for similar changes, you can first submit a draft to describe the idea and set up the design plan, and then implement the code.

In fact, the idea behind this PR is that I saw an article that introduced how to save some information of the stack. When using it again, there is no need to call the runtime library again to obtain it. I implemented it locally, and the code In fact, it's not much, only about forty or fifty lines (the local version is twice as fast as the current performance), so I just want to try gerror to see if it can be implemented. At present, what can be achieved is the same as the previous one. The effect is exactly the same, no difficulty This idea can actually be used in other places, such as glog. If you need to print the file name, function name and line number, you can also obtain it once and reuse it later, with almost no cost.

It is recommended to carry out a transformation of glog, which is used in more places

good idea

gqcn · 2024-12-25T13:37:02Z

@wln32 堆栈降噪那个是很有必要的，因为Wrap后的新error必然会存在重复堆栈，顶部的错误会包含所有的堆栈。至于性能那个，我觉得思路挺不错的，这个思路也很简单。但是这个实现的代码，老实说，有点看不明白啊，我花很多时间去review都看不明白，我有点头疼。能否让AI来优化一下代码，使得实现代码更优雅易维护一些。

Issues-translate-bot · 2024-12-25T13:37:12Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@wln32 Stack noise reduction is very necessary, because the new error after Wrap will inevitably have duplicate stacks, and the top error will contain all stacks. As for performance, I think the idea is pretty good and very simple. But to be honest, the code for this implementation is a bit hard to understand. I spent a lot of time reviewing it and couldn’t understand it. It gave me a headache. Can AI be used to optimize the code to make the code more elegant and easier to maintain?

wln32 added 5 commits December 20, 2024 15:43

添加缓存

dacf7d1

更新缓存逻辑

d9a5d54

逻辑优化

d58355c

test

3228484

up

c54232b

wln32 commented Dec 21, 2024

View reviewed changes

wln32 added 3 commits December 22, 2024 15:52

优化缓存逻辑，去除unsafe

30d1ee2

删除测试

a88d871

up

adf8db4

wln32 added the enhancement label Dec 22, 2024

修复非gerror.Error无限循环的错误

5432b87

gogf deleted a comment from Issues-translate-bot Dec 22, 2024

wln32 and others added 2 commits December 22, 2024 19:55

up

34e6d3d

Merge branch 'master' into perf/gerror

fcef9db

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf(errors/gerror): 缓存堆栈信息 #4071

perf(errors/gerror): 缓存堆栈信息 #4071

wln32 commented Dec 21, 2024 •

edited

Loading

wln32 Dec 21, 2024

gqcn commented Dec 23, 2024

Issues-translate-bot commented Dec 23, 2024

wln32 commented Dec 23, 2024

Issues-translate-bot commented Dec 23, 2024

houseme commented Dec 24, 2024

Issues-translate-bot commented Dec 24, 2024

wln32 commented Dec 24, 2024

Issues-translate-bot commented Dec 24, 2024

gqcn commented Dec 25, 2024

Issues-translate-bot commented Dec 25, 2024

		// You can obtain one at https://github.com/gogf/gf.

		package gerror_test

perf(errors/gerror): 缓存堆栈信息 #4071

Are you sure you want to change the base?

perf(errors/gerror): 缓存堆栈信息 #4071

Conversation

wln32 commented Dec 21, 2024 • edited Loading

gf的输出结果为

此次pr的输出结果

性能比对

当前主分支的new

当前主分支的wrap

当前pr的new

当前pr的wrap

wln32 Dec 21, 2024

Choose a reason for hiding this comment

gqcn commented Dec 23, 2024

Issues-translate-bot commented Dec 23, 2024

wln32 commented Dec 23, 2024

Issues-translate-bot commented Dec 23, 2024

houseme commented Dec 24, 2024

Issues-translate-bot commented Dec 24, 2024

wln32 commented Dec 24, 2024

Issues-translate-bot commented Dec 24, 2024

gqcn commented Dec 25, 2024

Issues-translate-bot commented Dec 25, 2024

wln32 commented Dec 21, 2024 •

edited

Loading