Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

A couple of thoughts for improvements #1

Open
ianhelle opened this issue May 20, 2022 · 0 comments
Open

A couple of thoughts for improvements #1

ianhelle opened this issue May 20, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@ianhelle
Copy link

For the last few versions we should automatically look for and use the VT key from settings - so you don't need to get it manually (I think our example notebook still does this - I will try to create some proper docs and update this)

# Get API key from mscticpyconfig, comment if you use the variable above
vt_key = get_provider_settings("TIProviders")["VirusTotal"].args["AuthKey"]

# Instantiate vt_lookup object
vt_lookup = VTLookupV3(vt_key)

We also do this in the init of VTLookupV3 if you're running in notebook so it shouldn't be needed

nest_asyncio.apply()

You should be able to use our IoCExtract module to do this

  # Regex
  validip = "^((25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\.){3}(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])$"
  validdomain = "^((?!-)[A-Za-z0-9-]" + "{1,63}(?<!-)\\.)" + "+[A-Za-z]{2,6}"
  validurl = r"(?i)\b((?:https?://|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'\".,<>?«»“”‘’]))"
  
  # Compile the ReGex
  domain = re.compile(validdomain)
  ip = re.compile(validip)
  url = re.compile(validurl)
  
  print("[+] Searching VT samples that have relationship with: " + keyword.value)
  
  if re.search(ip, keyword.value):
      domain_relation = vt_lookup.lookup_io....

Using IoCExtract

   from msticpy.sectools.ioc_extract import IoCExtract
   # note this location will change in MP V2.0 to msticpy.transform.ioc_extract
   ioc_extract = IoCExtract()

  if ioc_extract.validate(keyword.value, "ipv4"):
      domain_relation = vt_lookup.lookup_io....
  if ioc_extract.validate(keyword.value, "dns"):
      domain_relation = vt_lookup.lookup_io....'
  if ioc_extract.validate(keyword.value, "url"):
      domain_relation = vt_lookup.lookup_io....'

IoCExtract and VT relations also support file hashes for domain relationships.
(you can use "file_hash" generically or a specific type: "md5_hash", "sha1_hash", "sha256_hash")

We also expose getting these relationships with some pivot functions:

from msticpy.datamodel.entities import IpAddress, Dns, Url

domain_rel_df = IpAddress.VT.downloaded_files(keyword.value)

Again, this not yet well documented but I hope to get to this soon. :-)

Also - the notebook display is being messed up - prob due to an html error. I've seen this before in notebooks on Github, which seems less tolerant of html errors.
I noticed these two lines have malformed closing tags, could be that.

display(HTML("<h2>Samples Hunting on domain/ip/url Relationship</h2"))
display(HTML("<h3>Usage: </h3"))
@fr0gger fr0gger added the enhancement New feature or request label May 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@fr0gger @ianhelle