tailscale/logtail: log-filtering #10531
Conversation
|
For this to make it into 1.56 it would need to be finished tomorrow. However the consumer, who shall remain nameless because this is a public PR, wouldn't be likely to deploy it in December anyway. If it slips into 1.58 instead that would be fine. |
|
At this point I'd request this go in after release 1.56 branches, to be included in 1.58. |
as2643
force-pushed
the
anishka/coral_log_filtering
branch
2 times, most recently
from
d003d42 to
f789d98
Compare
as2643
changed the title
as2643
force-pushed
the
anishka/coral_log_filtering
branch
3 times, most recently
from
15d10aa to
ffc924b
Compare
as2643
force-pushed
the
anishka/coral_log_filtering
branch
from
d9a5abe to
cf8e5df
Compare
logtail/logtail.go
Outdated
| func redactIPs(buf []byte) []byte { | ||
| regexMatchesIPv6 := regexp.MustCompile(`([0-9a-fA-F]{1,4}):([0-9a-fA-F]{1,4}):([0-9a-fA-F:]{1,4})*`) | ||
| regexMatchesIPv4 := regexp.MustCompile(`(\d{1,3})\.(\d{1,3})\.\d{1,3}\.\d{1,3}`) |
There was a problem hiding this comment.
We should bump these variables up to the package level so we don't re-compile the patterns each time it's called.
| func redactIPs(buf []byte) []byte { | |
| regexMatchesIPv6 := regexp.MustCompile(`([0-9a-fA-F]{1,4}):([0-9a-fA-F]{1,4}):([0-9a-fA-F:]{1,4})*`) | |
| regexMatchesIPv4 := regexp.MustCompile(`(\d{1,3})\.(\d{1,3})\.\d{1,3}\.\d{1,3}`) | |
| var ( | |
| regexMatchesIPv6 = regexp.MustCompile(`([0-9a-fA-F]{1,4}):([0-9a-fA-F]{1,4}):([0-9a-fA-F:]{1,4})*`) | |
| regexMatchesIPv4 = regexp.MustCompile(`(\d{1,3})\.(\d{1,3})\.\d{1,3}\.\d{1,3}`) | |
| ) | |
| func redactIPs(buf []byte) []byte { |
logtail/logtail.go
Outdated
| @@ -757,6 +767,33 @@ func (l *Logger) Write(buf []byte) (int, error) { | |||
| return inLen, err | |||
| } | |||
|
|
|||
| func redactIPs(buf []byte) []byte { | |||
There was a problem hiding this comment.
Although it's unexported, this function is doing enough interesting work that it probably deserves a doc comment. In particular it'd be good to say what the input is and to summarize how the function transforms it.
logtail/logtail.go
Outdated
| regexMatchesIPv6 := regexp.MustCompile(`([0-9a-fA-F]{1,4}):([0-9a-fA-F]{1,4}):([0-9a-fA-F:]{1,4})*`) | ||
| regexMatchesIPv4 := regexp.MustCompile(`(\d{1,3})\.(\d{1,3})\.\d{1,3}\.\d{1,3}`) | ||
|
|
||
| out := regexMatchesIPv6.ReplaceAllStringFunc(string(buf), func(s string) string { |
There was a problem hiding this comment.
I would guess we can save some allocation and copying if we use the byte forms:
regexMatchesIPv6.ReplaceAllFunc(buf, func(b []byte) []byte {
// ...
prefix := bytes.Split(b, []byte(":"))
return bytes.Join(append(prefix[:2], []byte("x")), []byte(":"))
})That way we don't need to convert to string and back, since we wanted the output as a byte slice anyway. But, YMMV.
…ilscaled. Updates #15664 Signed-off-by: Anishka Singh <anishkasingh66@gmail.com>
as2643
force-pushed
the
anishka/coral_log_filtering
branch
from
4a38dfc to
9faf6f3
Compare
…ilscaled. (tailscale#10531) Updates #15664 Signed-off-by: Anishka Singh <anishkasingh66@gmail.com>
In order to provide more visibility into the operation of tailnets for a specific customer, redact public IP addresses such as employee homes from the log content before the upload.
Updates #15664