Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat] Windows Module add wmi metricset #42017

Merged
merged 82 commits into from
Jan 30, 2025
Merged
Changes from 1 commit
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
5aff55e
Add stub for wmi module
herrBez Dec 3, 2024
60dfc6d
Execute mage update
herrBez Dec 3, 2024
4c6334f
Add first draft of wmi metricset
herrBez Dec 3, 2024
e5e45a5
Merge branch 'main' of https://github.com/elastic/beats into wmi
herrBez Dec 12, 2024
880a7ef
Merge branch 'main' of https://github.com/elastic/beats into wmi
herrBez Dec 12, 2024
ebc8ceb
Add first draft of wmi windows module
herrBez Dec 12, 2024
6198252
Add config unit test
herrBez Dec 12, 2024
98357b7
Add unit test for the config class
herrBez Dec 12, 2024
70abaef
Add license notice mage fmt
herrBez Dec 12, 2024
0349726
Add License to wmi/wmi.go
herrBez Dec 12, 2024
65d594f
Update example
herrBez Dec 12, 2024
aface56
Use wrapping fromat verb for fmt.Errorf
herrBez Dec 13, 2024
b833f93
Fix mispelled comment
herrBez Dec 13, 2024
a5e69f2
Add first draft of the documentation
herrBez Dec 13, 2024
18ba51d
Add the config reference and config file
herrBez Dec 13, 2024
dfb1338
Bump microsfot/wmi to 0.25.1
herrBez Dec 13, 2024
aa6b937
Make sure the wmi metricset is only used on windows
herrBez Dec 13, 2024
de36999
Run mage update
herrBez Dec 13, 2024
64201dc
Add License for microsoft/wmi library in Notice.txt
herrBez Dec 13, 2024
c5adc90
Add Timeout configuration
herrBez Dec 16, 2024
d706121
Introduce the ExecuteGuardedQueryInstances to wait for at most a time…
herrBez Dec 16, 2024
e0477e5
Merge branch 'wmi' of github.com:herrBez/beats into wmi
herrBez Dec 16, 2024
59b625c
Add invokation of CloseAllInstances() to make sure to free resources …
herrBez Dec 21, 2024
f01ba97
Add license header to utils.go
herrBez Dec 23, 2024
0a6c670
Rename timeout to warning_threshold to conceive the message that the …
herrBez Dec 24, 2024
06ae564
Refactor the ExecuteGuardedQueryInstances to use the context.WithTimeout
herrBez Dec 24, 2024
c1c956b
Add unit test for ExecuteGuardedQueryInstances
herrBez Dec 24, 2024
b29f5c7
Add parameter IncludeEmptyString. Create function to check skip condi…
herrBez Dec 24, 2024
bf9c088
Add namespace at the query level and add a structure to index queries…
herrBez Dec 24, 2024
c54eca0
Rename the config method to be more explicit
herrBez Dec 24, 2024
3e232f2
Add Primitives to deal with the type convrsion for strings
herrBez Dec 24, 2024
7ff8b88
Add unit test for the conversion function
herrBez Dec 24, 2024
f1e67e4
Add heuristic to determine if fetching the CIMType is needed
herrBez Dec 24, 2024
c4f7101
Add type conversion
herrBez Dec 24, 2024
1e00ac9
Improve comments to explicitly state what are the config parameters u…
herrBez Dec 24, 2024
54e6642
Add license header to wmi and utils test
herrBez Dec 24, 2024
0cced56
Run mage fmt
herrBez Dec 24, 2024
9661c45
Update the reference config after the final implementation
herrBez Dec 24, 2024
ab3c5e6
Remove the dummy field definition
herrBez Dec 24, 2024
6c3c55f
Add sample data in the data.json file
herrBez Dec 24, 2024
969b7cc
Fix go.mod and change NOTICE.txt to reflect the fact that go-ole is n…
herrBez Dec 24, 2024
be95605
Make sure that the wmi tests run only on windows
herrBez Dec 24, 2024
9a8b9b8
Improve the error message to prepare a Troubleshooting Guide
herrBez Dec 24, 2024
082873a
Merge branch 'wmi' of github.com:herrBez/beats into wmi
herrBez Dec 24, 2024
e7fcdd4
Merge remote-tracking branch 'upstream/main' into wmi
herrBez Jan 15, 2025
4882238
Fix test to use right function
herrBez Jan 15, 2025
213d92e
Add doc.go file
herrBez Jan 15, 2025
83e2627
Add pragma to avoid compiling utils.go on platforms other than windows
herrBez Jan 15, 2025
f7b4865
Run make update BEATS=metricbeat
herrBez Jan 15, 2025
8ff47b4
Use Metricset Logger instead of generic one in wmi.go
herrBez Jan 16, 2025
cd513ef
Make sure that we are using the Metricset logger everywhere. Address …
herrBez Jan 16, 2025
f4c0c7d
Merge branch 'wmi' of github.com:herrBez/beats into wmi
herrBez Jan 16, 2025
8ccb2dc
Merge branch 'main' into wmi
herrBez Jan 16, 2025
df56ec6
Run make updates BEATS=metricbeat
herrBez Jan 16, 2025
cf9b2f6
Get rid of duplicated rawResult.Clear()
herrBez Jan 16, 2025
63a35c7
Merge branch 'main' into wmi
herrBez Jan 28, 2025
5fd9ba1
Review: remove redundant multiplication by time.Second
herrBez Jan 29, 2025
e075995
Review: Fix grammar of error
herrBez Jan 29, 2025
20e260c
Review: Get rid of the panic
herrBez Jan 29, 2025
f734e96
Review: Remove redundant check
herrBez Jan 29, 2025
10a9310
Review: Make the namespace settings easier to understand
herrBez Jan 29, 2025
b01e730
Review: Honor the query-level namespace in the output document and mo…
herrBez Jan 29, 2025
7617f03
Add a warning if a given query is not producing results
herrBez Jan 29, 2025
be75e47
Fix golint warning
herrBez Jan 29, 2025
500b39c
Merge branch 'main' into wmi
herrBez Jan 29, 2025
e6117c2
add results of 'make update'
tommyers-elastic Jan 29, 2025
dfc331b
Merge branch 'main' into wmi
tommyers-elastic Jan 29, 2025
5e86d2d
Merge branch 'main' into wmi
ishleenk17 Jan 30, 2025
b6d2967
Add changelog entry for the wmi metricset
herrBez Jan 30, 2025
f766c51
Fix pull request number
herrBez Jan 30, 2025
e076caa
Add an explicit variable for the converted field and double check if …
herrBez Jan 30, 2025
35304c5
Make warning easier to read
herrBez Jan 30, 2025
07fb897
Add empty newline in the config.yml
herrBez Jan 30, 2025
2ab90d0
Add codeowners entry for wmi
herrBez Jan 30, 2025
34a7dc9
Add windows.yml.disabled
herrBez Jan 30, 2025
ec5ca4f
Modify codeowners at the dataset level
herrBez Jan 30, 2025
e26133a
Fix test after renaming of the error message
herrBez Jan 30, 2025
ae7f2ae
Update metricbeat/docs/modules/windows.asciidoc
herrBez Jan 30, 2025
ca8224d
Fix config reference to generate correct doc
herrBez Jan 30, 2025
ef48f83
Merge branch 'main' into wmi
herrBez Jan 30, 2025
845c91b
Update xpack metricbeat.reference.yml
herrBez Jan 30, 2025
e88f7af
Merge branch 'wmi' of github.com:herrBez/beats into wmi
herrBez Jan 30, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add Primitives to deal with the type convrsion for strings
  • Loading branch information
herrBez committed Dec 24, 2024
commit 3e232f2b96213823af3c8fca4e44997015ec2ef0
103 changes: 101 additions & 2 deletions metricbeat/module/windows/wmi/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,112 @@ package wmi
import (
"context"
"fmt"
"strconv"
"time"

wmi "github.com/microsoft/wmi/pkg/wmiinstance"

"github.com/elastic/elastic-agent-libs/logp"
"github.com/go-ole/go-ole"
"github.com/go-ole/go-ole/oleutil"
base "github.com/microsoft/wmi/go/wmi"
wmi "github.com/microsoft/wmi/pkg/wmiinstance"
)

// Utilities related to Type conversion

// WmiStringConversionFunction defines a function type for converting string values
// into other data types, such as integers or timestamps.
type WmiStringConversionFunction func(string) (interface{}, error)

func ConvertUint64(v string) (interface{}, error) {
return strconv.ParseUint(v, 10, 64)
}

func ConvertSint64(v string) (interface{}, error) {
return strconv.ParseInt(v, 10, 64)
}

func ConvertDatetime(v string) (interface{}, error) {
layout := "20060102150405.999999-0700"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we put this in a constant with a proper name?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and how come the +"0" is outside the layout sttring?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The WMI time strings are formatted 20060102150405.999999-070 (they only have three digits for the timezone).

By looking at the documentation https://pkg.go.dev/time#pkg-constants I thought we were forced to have either 4 or 2 digits, while we have three in the string. So I thought to reduce the case to 4 digits.

It seems to work also by using 3 digits. Should I go with three digits?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeh i think you can any format you want, so let's go with 3 if it works.

return time.Parse(layout, v+"0")
}

func ConvertString(v string) (interface{}, error) {
return v, nil
}

// Given a Property it returns its CIM Type Qualifier
// https://learn.microsoft.com/en-us/windows/win32/wmisdk/cimtype-qualifier
// We assume that it is **always** defined for every property to simiplifying
tommyers-elastic marked this conversation as resolved.
Show resolved Hide resolved
// The error handling
func getPropertyType(property *ole.IDispatch) base.WmiType {
rawType := oleutil.MustGetProperty(property, "CIMType")

value, err := wmi.GetVariantValue(rawType)
if err != nil {
panic("Error retrieving the wmi property type")
herrBez marked this conversation as resolved.
Show resolved Hide resolved
}

return base.WmiType(value.(int32))
}

// Returns the "raw" SWbemProperty containing type information for a given field.
//
// The microsoft/wmi library does not have a function that given an instance and a property name
// returns the wmi.wmiProperty object. This function mimics the behavior of the `GetSystemProperty`
// method in the wmi.wmiInstance struct and applies it on the Properties_ field
// https://github.com/microsoft/wmi/blob/v0.25.2/pkg/wmiinstance/WmiInstance.go#L87
//
// Note: We are not instantiating a wmi.wmiProperty because of this issue
// https://github.com/microsoft/wmi/issues/150
// Once this issue is resolved, we can instantiate a wmi.WmiProperty and eliminate
// the need for the "getPropertyType" function.
func getProperty(instance *wmi.WmiInstance, propertyName string) (*ole.IDispatch, error) {
// Documentation: https://learn.microsoft.com/en-us/windows/win32/wmisdk/swbemobject-properties-
rawResult, err := oleutil.GetProperty(instance.GetIDispatch(), "Properties_")
if err != nil {
return nil, err
}

// SWbemObjectEx.Properties_ returns
// an SWbemPropertySet object that contains the collection
// of properties for the c class
sWbemObjectExAsIDispatch := rawResult.ToIDispatch()
defer rawResult.Clear()

// Get the property
sWbemProperty, err := oleutil.CallMethod(sWbemObjectExAsIDispatch, "Item", propertyName)
if err != nil {
return nil, err
}

return sWbemProperty.ToIDispatch(), nil
}

// Given an instance and a property Name, it returns the appropriate conversion function
func GetConvertFunction(instance *wmi.WmiInstance, propertyName string) (WmiStringConversionFunction, error) {
rawProperty, err := getProperty(instance, propertyName)
if err != nil {
return nil, err
}
propType := getPropertyType(rawProperty)

var f WmiStringConversionFunction

switch propType {
case base.WbemCimtypeDatetime:
f = ConvertDatetime
case base.WbemCimtypeUint64:
f = ConvertUint64
case base.WbemCimtypeSint64:
f = ConvertSint64
default: // For all other types we return the identity function
f = ConvertString
}
return f, err
}

// Utilities related to Warning Threshold

// Define an interface to allow unit-testing long-running queries
// *wmi.wmiSession is an implementation of this interface
type WmiQueryInterface interface {
Expand Down
Loading