Skip to content

Commit

Permalink
Added cache and throttling for certificates, keychain_acls, and `…
Browse files Browse the repository at this point in the history 
…keychain_items` tables. (osquery#8192)

Fixes osquery#7780 
Related issue: fleetdm/fleet#13065

Adding a cache for macOS keychain file accesses.

The cache checks whether a keychain file has been modified by comparing the file's SHA256 hash. If the file has been modified, the cache also checks whether the file has been accessed recently. If it has been accessed within the configured interval, the old cached results are returned.

The cache works independently for each table. This means that multiple tables can access the keychain files within the interval, but each one of them can only do so once.

The following feature flags have been added:
```
    --keychain_access_cache                          Use a cache for keychain accesses (default true)
    --keychain_access_interval VALUE                 Minimum minutes required between keychain accesses. Keychain cache must be enabled to use
```

Default `keychain_access_interval` is 5 minutes.

Old table results exactly match new table results when ordered by primary key.

Performance results for `certificates` for 10 rounds (old vs new):
```
 U:1  C:0  M:3  F:0  D:0  manual: utilization: 13.940000000000001 cpu_time: 0.14197365480000002 memory: 35979264.0 fds: 4.0 duration: 0.5259468078613281
 U:1  C:0  M:3  F:0  D:0  manual: utilization: 13.236666666666668 cpu_time: 0.13929492440000002 memory: 35386163.2 fds: 4.0 duration: 0.5783782720565795
```

Performance results for `certificates` for 10 counts (new is faster and less memory due to cache):
```
 U:2  C:1  M:3  F:0  D:2  manual: utilization: 44.29999999999999 cpu_time: 0.669130928 memory: 50331648.0 fds: 4.0 duration: 1.5303008556365967
 U:2  C:1  M:3  F:0  D:2  manual: utilization: 28.366666666666664 cpu_time: 0.42878788 memory: 40534016.0 fds: 4.0 duration: 1.0357069969177246
```

Performance results for `keychain_acls` for 10 rounds (old vs new):
```
 U:1  C:0  M:3  F:0  D:0  manual: utilization: 10.57 cpu_time: 0.10779758619999999 memory: 29374873.6 fds: 4.0 duration: 0.5254422664642334
 U:1  C:0  M:3  F:0  D:0  manual: utilization: 11.366666666666669 cpu_time: 0.1201387166 memory: 29420748.8 fds: 4.0 duration: 0.5800627708435059
```

Performance results for `keychain_acls` for 10 counts (new is faster and less memory due to cache):
```
 U:2  C:1  M:3  F:0  D:2  manual: utilization: 22.059999999999995 cpu_time: 0.557502384 memory: 77463552.0 fds: 4.0 duration: 2.0405030250549316
 U:2  C:1  M:3  F:0  D:2  manual: utilization: 26.733333333333334 cpu_time: 0.405785928 memory: 36782080.0 fds: 4.0 duration: 1.0386288166046143
```

Performance results for `keychain_items` for 10 rounds (old vs new). **New performance is better.** This is likely because the new code only opens each keychain file once, while old code opened each keychain file multiple times -- once for each keychain item type (password, certificate, etc.)
```
 U:2  C:1  M:3  F:0  D:1  manual: utilization: 30.510833333333334 cpu_time: 0.45226203760000006 memory: 29961420.8 fds: 4.0 duration: 0.9804916620254517
 U:2  C:0  M:3  F:0  D:0  manual: utilization: 20.805 cpu_time: 0.2112246234 memory: 26363494.4 fds: 4.0 duration: 0.5286885976791382
```

Performance results for `keychain_items` for 10 counts (new is way faster and less memory):
```
 U:3  C:2  M:3  F:0  D:3  manual: utilization: 78.25999999999999 cpu_time: 3.946559488 memory: 41320448.0 fds: 4.0 duration: 4.564563989639282
 U:2  C:1  M:3  F:0  D:2  manual: utilization: 25.474999999999998 cpu_time: 0.511153552 memory: 33996800.0 fds: 4.0 duration: 1.5302011966705322
```
  • Loading branch information
@getvictor
getvictor authored Dec 15, 2023
1 parent f972f69 commit 6f380fc
Show file tree
Hide file tree
Showing 11 changed files with 581 additions and 93 deletions.
1 change: 1 addition & 0 deletions osquery/tables/system/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,7 @@ function(generateOsqueryTablesSystemSystemtable)
PROPERTIES ENVIRONMENT "TEST_CONF_FILES_DIR=${TEST_CONFIGS_DIR}"
)
elseif(DEFINED PLATFORM_MACOS)
add_test(NAME osquery_tables_system_darwin_keychain_tests-test COMMAND osquery_tables_system_darwin_keychain_tests-test)
add_test(NAME osquery_tables_system_darwin_tests-test COMMAND osquery_tables_system_darwin_tests-test)

set_tests_properties(
Expand Down
155 changes: 131 additions & 24 deletions osquery/tables/system/darwin/certificates.mm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@
namespace osquery {
namespace tables {

// The table key for Keychain cache access.
static const KeychainTable KEYCHAIN_TABLE = KeychainTable::CERTIFICATES;

void genCertificate(X509* cert, const std::string& path, QueryData& results) {
Row r;

Expand Down Expand Up @@ -136,6 +139,9 @@ void genFileCertificate(const std::string& path, QueryData& results) {
QueryData genCerts(QueryContext& context) {
QueryData results;

// Lock keychain access to 1 table/thread at a time.
std::unique_lock<decltype(keychainMutex)> lock(keychainMutex);

// Allow the caller to set both an explicit keychain search path
// and certificate files on disk.
std::set<std::string> keychain_paths;
Expand All @@ -159,49 +165,150 @@ QueryData genCerts(QueryContext& context) {
}));

@autoreleasepool {
// Map of path to hash and keychain reference. This ensures we don't open
// the same keychain multiple times when the table's path constraint is
// used.
std::map<std::string,
std::tuple<boost::filesystem::path, std::string, SecKeychainRef>>
opened_keychains;
if (!paths.empty()) {
for (const auto& path : paths) {
// Check whether path is valid
boost::system::error_code ec;
auto source =
boost::filesystem::canonical(boost::filesystem::path(path), ec);
if (ec.failed() || !is_regular_file(source, ec) || ec.failed()) {
TLOG << "Could not access file " << path << " " << ec.message();
continue;
}

// Check cache
bool err = false;
std::string hash;
bool hit =
keychainCache.Read(source, KEYCHAIN_TABLE, hash, results, err);
if (err) {
TLOG << "Could not read the file at " << path << "" << ec.message();
continue;
}
if (hit) {
continue;
}

SecKeychainRef keychain = nullptr;
SecKeychainStatus keychain_status;
auto status = SecKeychainOpen(path.c_str(), &keychain);
if (status != errSecSuccess || keychain == nullptr ||
SecKeychainGetStatus(keychain, &keychain_status) != errSecSuccess) {
OSStatus status;
OSQUERY_USE_DEPRECATED(status =
SecKeychainOpen(path.c_str(), &keychain));
bool genFileCert = false;
if (status != errSecSuccess || keychain == nullptr) {
genFileCert = true;
} else {
OSQUERY_USE_DEPRECATED(
status = SecKeychainGetStatus(keychain, &keychain_status));
if (status != errSecSuccess) {
genFileCert = true;
}
}
if (genFileCert) {
if (keychain != nullptr) {
CFRelease(keychain);
}
genFileCertificate(path, results);
QueryData new_results;
genFileCertificate(path, new_results);
// Write new results to the cache.
keychainCache.Write(source, KEYCHAIN_TABLE, hash, new_results);
results.insert(results.end(), new_results.begin(), new_results.end());
} else {
// This path will be re-accessed later.
keychain_paths.insert(path);
CFRelease(keychain);
opened_keychains.insert(
{path, std::make_tuple(source, hash, keychain)});
}
}
} else {
for (const auto& path : kSystemKeychainPaths) {
keychain_paths.insert(path);
}
auto homes = osquery::getHomeDirectories();
for (const auto& dir : homes) {
for (const auto& keychains_dir : kUserKeychainPaths) {
keychain_paths.insert((dir / keychains_dir).string());
keychain_paths = getKeychainPaths();
}

// Since we are used a cache for each keychain file, we must process
// certificates one keychain file at a time.
std::set<std::string> expanded_paths = expandPaths(keychain_paths);
for (const auto& path : expanded_paths) {
SecKeychainRef keychain = nullptr;
std::string hash;
boost::filesystem::path source;
auto it = opened_keychains.find(path);
if (it != opened_keychains.end()) {
source = std::get<0>(it->second);
hash = std::get<1>(it->second);
keychain = std::get<2>(it->second);
} else {
// Check whether path is valid
boost::system::error_code ec;
source =
boost::filesystem::canonical(boost::filesystem::path(path), ec);
if (ec.failed() || !is_regular_file(source, ec) || ec.failed()) {
// File does not exist or user does not have access. Don't log here to
// reduce noise.
continue;
}

// Check cache
bool err = false;
bool hit =
keychainCache.Read(source, KEYCHAIN_TABLE, hash, results, err);
if (err) {
TLOG << "Could not read the file at " << source.string() << ""
<< ec.message();
continue;
}
if (hit) {
continue;
}

// Cache miss. We need to generate new results.
OSStatus status;
OSQUERY_USE_DEPRECATED(status =
SecKeychainOpen(source.c_str(), &keychain));
if (status != errSecSuccess || keychain == nullptr) {
if (keychain != nullptr) {
CFRelease(keychain);
}
// Cache an empty result to prevent the above API call in the future.
keychainCache.Write(source, KEYCHAIN_TABLE, hash, {});
continue;
}
}
}

// Keychains/certificate stores belonging to the OS.
CFArrayRef certs =
CreateKeychainItems(keychain_paths, kSecClassCertificate);
// Must have returned an array of matching certificates.
if (certs != nullptr) {
if (CFGetTypeID(certs) == CFArrayGetTypeID()) {
auto certificate_count = CFArrayGetCount(certs);
for (CFIndex i = 0; i < certificate_count; i++) {
auto cert = (SecCertificateRef)CFArrayGetValueAtIndex(certs, i);
genKeychainCertificate(cert, results);
auto keychains = CFArrayCreateMutable(nullptr, 1, &kCFTypeArrayCallBacks);
CFArrayAppendValue(keychains, keychain);
QueryData new_results;
// Keychains/certificate stores belonging to the OS.
CFArrayRef certs = CreateKeychainItems(keychains, kSecClassCertificate);
CFRelease(keychains);
// Must have returned an array of matching certificates.
if (certs != nullptr) {
if (CFGetTypeID(certs) == CFArrayGetTypeID()) {
auto certificate_count = CFArrayGetCount(certs);
for (CFIndex i = 0; i < certificate_count; i++) {
auto cert = (SecCertificateRef)CFArrayGetValueAtIndex(certs, i);
genKeychainCertificate(cert, new_results);
}
}
CFRelease(certs);
keychainCache.Write(source, KEYCHAIN_TABLE, hash, new_results);
results.insert(results.end(), new_results.begin(), new_results.end());
} else {
// Cache an empty result to prevent the above API call in the future.
keychainCache.Write(source, KEYCHAIN_TABLE, hash, {});
}
CFRelease(certs);
}
}

if (FLAGS_keychain_access_cache) {
TLOG << "Total Keychain Cache entries: " << keychainCache.Size();
}

return results;
}
}
Expand Down
50 changes: 48 additions & 2 deletions osquery/tables/system/darwin/keychain.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
#include <CoreFoundation/CoreFoundation.h>
#include <Security/Security.h>

#include <osquery/core/flags.h>
#include <osquery/core/tables.h>

namespace osquery {
Expand All @@ -23,11 +24,56 @@ namespace tables {
extern const std::vector<std::string> kSystemKeychainPaths;
extern const std::vector<std::string> kUserKeychainPaths;

void genKeychains(const std::string& path, CFMutableArrayRef& keychains);
// Declare keychain flags. They are defined in keychain_utils.cpp.
DECLARE_bool(keychain_access_cache); // enable flag
DECLARE_uint32(keychain_access_interval); // throttling flag

// The tables supported by Keychain Cache
enum class KeychainTable { CERTIFICATES, KEYCHAIN_ACLS, KEYCHAIN_ITEMS };

// The KeychainCache caches results associated with keychain files,
// and throttles access to these files.
class KeychainCache {
private:
// KeychainCacheEntry contains cache metadata and cached results
// for a single keychain file.
class KeychainCacheEntry {
public:
std::chrono::system_clock::time_point timestamp; // time of last access
std::string hash; // sha256 keychain file hash
QueryData results; // the cached results
};
std::map<std::pair<boost::filesystem::path, KeychainTable>,
KeychainCacheEntry>
cache;

public:
// Read checks the hash and returns 1 for a cache hit or 0 for a cache miss.
// If hit, results are populated. hash is the file hash
bool Read(const boost::filesystem::path& path,
KeychainTable table,
std::string& hash,
QueryData& results,
bool& err);
// Write a cache entry.
void Write(const boost::filesystem::path& path,
KeychainTable table,
const std::string& hash,
const QueryData& results);
size_t Size() {
return cache.size();
}
};
extern KeychainCache keychainCache;
extern std::mutex keychainMutex;

// Expand paths to individual files
std::set<std::string> expandPaths(const std::set<std::string>& paths);

std::string getKeychainPath(const SecKeychainItemRef& item);

/// Generate a list of keychain items for a given item type.
CFArrayRef CreateKeychainItems(const std::set<std::string>& paths,
CFArrayRef CreateKeychainItems(CFMutableArrayRef keychains,
const CFTypeRef& item_type);

std::set<std::string> getKeychainPaths();
Expand Down
Loading

0 comments on commit 6f380fc

Please sign in to comment.