Posted on
by

Auditing all the things: The future of smarter monitoring and detection

Jen Andre
Founder @threatstack
@fun_cuddles
  • Started with question on twitter:
    • Can you produce a list of all process running on your network?
    • But then expanded… wanted to know everything
  • Why? Is there a reason to be this paranoid?
    • prevention fails. 
  • should you care?
    • if you’re a startup about pets and you get hacked, you just change all passwords
    • but if you’re a pharmaceutical company, then you really do care. 
  • “We found no evidence that any customer data was accessed, changed or lost”
    • Did you look for evidence?
    • Do you really know what happened?
    • If you log everything (the right things), then you don’t have to do forensic evidence.
  • “We’re in the cloud!”
  • Continuous security monitoring
    • auditing + analytics + automation
  • Things to monitor:
    • Systems: authentications, process activity, network activity, kernel modules, file system
    • Apps: authentications, db requests, http requests
    • services: AWS api calls, SaaS api calls
  • In order to do:
    • Intrusion detection
    • “active defense”
    • rapid incident response
  • “Use the host, Luke”
  • apt-get install audit
    • pros:
      • super powerful
      • build into your kernel
      • relatively low overhead
    • you can audit logins, system calls.
  • auditd
    • the workings:
      • userland audit daemon and tools <- link="" net="" socket=""> kernel thread queue <- audit="" doing="" kernel="" li="" messages="" things="" threads="">
      • /var/log/audit

    • not so nice:
      • obtuse logging
      • enable rate limiting or it could ‘crash’ your box
        • auditctl -b 1000 -r 1500 # 100 buffers, 15000 eps max)
  • alternative: connect directly to net link socket and write your own audit listening
    • wrote a JSON format exporter
    • luajit! for filtering, transformation & alerting
  • authentications
    • who is logging in and from where?
    • Can use wtmp
      • can turn into json
    • auditd also returns login information
    • pam_loginid will add a session id to all executed commands so you can link real user to sudo’d commands
  • Detecting attacks
    • most often a long time goes by before people are hacked, sometimes years.
    • often they get a phone call from the government: hey, you’ve got servers sending data to china.
    • the hardest attack to detect is when the attacker is using valid credentials to access it.
    • things to think about:
      • is that user running commands he should;’t be?
        • ex: why is anyone except chef user running gcc on a production system?
      • why is a server that only accepts inbound connections suddenly making outbound ones?
        • or why connecting to machines other than expected ones?
      • are accounts logging in from unexpected locations? (or at unexpected times)
      • are files being copied to /lib /bin, etc.
  • Now go and audit!