This solution is in place at 2 former clients...
Best-practice methodology is at the core...
What you are about to read, is DevSecOps in action.
Taking things a step further by adapting designs to requirements, budgets, in-house capability and weighing up the importance of each service.
Working closely with vendors to ensure that this solution is transparent and compatible with their products.
The final result is a robust, scalable, highly fault tolerant SIEM, adaptive to additional and future demands, with much more flexibility than what your traditional SIEM deployment consists of.
Take a look below to see what the usual SIEM consists of ...
Agents to Manage: Getting the agents out there is the first problem, and when they are out there, what about config, management and upgrades? It's a battle that will never be over.
Single Syslog Stream: Feeding syslog streams direct from appliances to the SIEM is a bad idea for a good number of reasons, which are covered below.
Retention: Many SIEM's rely on their original design specification to define how much storage is available to have data ready for analysis, cold searching and archiving. This can of course end in disaster, we will talk about this later.
The three points being raised here are actually the most important points to address. Two of these points regard ingest of data and the final point regards the very lifecycle of the data post-processing.
There is no future for this topology without ripping out the guts sooner rather than later.
This Solution ...
Is tried, tested and proven effective, time after time. This certainly is no tin-pot solution...
This solution can actually come at zero cost where software licensing is concerned.
We provide the fastest known on-boarding of log sources using this solution. Thousands of Windows endpoints can be seamlessly on-boarded in to your SIEM quicker than ever before, with some simple GPO changes.
Additional to this, with the use of a Golden Copy store for logs, you are able to retain what you normally consider spam, to later be replayed to the SIEM in the event that these logs are required for further analysis.
Golden Copies: No more worrying about Licensing & Retention, and Operational Benefits explained ...
There are a number of hurdles (or limitations) when it comes to SIEM log retention and ingest licensing.
- Original Un-modified Copies: Many SIEMs will modify logs to trim bits off, such as remove PII, or discard part of a transaction.
- Confidentiality: Certain logs may just require retention and may also need to be secured from prying eyes, requiring minimal to zero analysis.
- Integrity: Events can be sent to a second place to ensure a fuller picture is retained, and can not be tampered with as easily.
- Availability: Disaster recovery with a complete dataset that has not been modified is a far-cry when all of your eggs are in one basket!
- If the SIEM is an appliance: You will need more appliances or pay for disk expansion, in any event, big retention means big bucks.
- If the SIEM resides as a software stack: You will need to think about major modification of the original design, particularly where you change your retention policy later on and made no room for scalable retention.
- SIEM Licensing differs: Some charge for daily ingest, some charge for overall retention.
- Log Spam: Nobody wants it, but sometimes it's helpful to build a fuller picture.
- APT Activity: Almost every time an Advanced Persistent Threat is exposed, we find ourselves looking back as far as we can, with significant gaps, to find the smallest of traces.
- SIEM Capacity Limitations: Isn't it useful to have every 4688/4689 Windows Event ID stored? Longer retention of DNS Data? This just isn't possible by normal intentions.
- Legaility & Compliance: Logs that have been processed by a SIEM generally are not admissible in a court, or can be easily defeated by a savvy defence/prosecution team.
- Frozen Data Archiving: This is possibly the most painful aspects of any log management system. Frozen data being inaccessible without overkill procedures to get it back, or compression causing searches to under-perform.
We may overcome these hurdles by having a golden copy store, which contains every log that has gone via the syslog hub or via the windows event collection system. The golden copy store is also able to replay events in to your SIEM for analysis, as and when required.
There are numerous benefits to having a Golden-Copy Store (AKA Litigation Hold). Most risks and constraints simply fail to exist after adoption
For the above points, we're able to instantly defeat those issues. There could be more points that spring to mind, from your experience.
Retain Spam & For Longer - Investigate Historical & APT Activity With Realism
ICMP or DNS requests, UDP traffic and local traffic logs, or system auth activity, may not be relevant today.
But what does the future hold? We know from experience that network traffic logs are often filtered out before being processed by a SIEM, this is the first place to start when you want to cut back on ingest/retention.
Not to forget the windows events, process creation/exit and the local object access events. They are desirable logs but not always considered essential for capture and retention for significant amounts of time.
Replay / Re-Load to SIEM
This is an extremely powerful feature of our solution. We know that using Splunk, you are able to set up a real-time streaming search that will re-analyze logs as they get pushed back in to the system from the Golden Copy Store. We will follow this claim up with a blog post in the near future. This is just one feature to find events like IOC's in real-time as the system is re-loaded.
Another important capability this brings, is the deep analysis of the data once it is back at rest. A huge dataset that is no longer available can easily be re-loaded in to a SIEM and queries run against it. This could be anything, an employee misconduct case involving web browsing history, DNS requests and site access logs. Or perhaps something far more serious...
Admissible Evidence in the Courts
Unless a SIEM is specifically designed and configured to provide solid evidence, this is a no-brainer, it's likely that the SIEM in question has a completely separate storage pool to store these logs in, in their original form, because it will be needed.
If a SIEM itself is providing the only source of logs, this could bring a lot of questions.
APT Activity
We here at SQEye.Net have investigated APT activity through mulitple facts provided in STIX feeds, SIEM content updates, and via studying threat actor reports provisioned by governments world-wide.
In every serious case, we had to reflect back on frozen logs and in some cases we have had to just take the hit - the logs weren't there. This is a very serious risk that can not be overlooked by any organisation with a SIEM operating.
With the Solarwinds hack, where we needed to load up 12 months worth of DNS logs, this didnt happen, we had the logs in our litigation hold and luckily, the litigation hold was around 14 months old at the time hence it contained that amount of data from all firewalls and DNS servers.
We searched the logs manually via the Golden Copy Store and re-loaded them in to the SIEM for further analysis by the security team members.
Off-loaded Searching
As mentioned in the above paragraph, it's useful to be able to search the logs without having to replay them to the SIEM. In many cases, you may just want to look for a DNS query, or an IP address, used for the C2.
Off-loaded searching is perfect for this purpose. Re-playing to the SIEM may have implications where you want to avoid busting your daily ingest license as this is only allowed 5 times per month! (With regards to Splunk)
A replay can also be time consuming in comparison to a basic search through the Golden Copy Store, or perhaps the SIEM itself does not harbor the space required to re-load the logs for simple searches?
How does this solution come at a low to zero cost?
Having a cost is purely optional, and here is why:
- We can use community based components to power this platform. Vendor support for the components is limited and usually community based, under the circumstances.
- You could purchase support for one or more components, rather than all.
- You could purchase premium editions of the components with full support contracts, this may be more suitable to higher budgets and is recommended, this could also be the case on compliance grounds.
Obviously, the work involved by ourselves will come at a cost, but the solution is wholly tailored to demands and in-line with your budget.
Additionally, you may already have a Windows or Linux support contract in place for your environment, in which case this would apply and that cost would already be covered.
What SIEM Vendors won't tell you!
Almost all SIEM vendors want you to be completely loyal to them by having agents peppered all over the place and locking you in to using their own developed protocols and standards. This gives you little flexibility.
They will never tell you about cheaper options, and they will never give you any sort of architecture where you can switch SIEM suppliers easily.
SIEM vendors may also pressure you in to buying their professional services.
SIEM vendors never admit that they don't fully understand your network, and the technology that it comprises of. This can be a huge issue when it comes to integration tasks.
Agentless SIEM
Using the built-in features of operating systems, logs can be natively ported to various destinations simultaneously.
This could be a local SIEM, a remote/local PCI log bunker, and an external SOC. That is 3 locations, a traditional setup would either bake your logs or be extremely limited when beaming to third parties. Your third party may have to host an identical SIEM at their end as a result.
All of this can be achieved with no SIEM agents being peppered everywhere!
The benefits of Agentless ...
- Less Management or Upgrades
- Minimal Deployment
- Smaller Overall Footprint
- Minimal Problems
- More flexibility - SIEM vendor change + easier to trial out a new SIEM or third party.
- Better third party support
- Auto-enroll Windows Logs
Enhanced Third-Party Support
Having third parties take a cut of logs is becoming quite the norm.
This may entail sending all logs for archiving purpose, or specific logs for additional screening such at threat intelligence.
The syslog hub plays the main part in this, where most/all logs are sent before being beamed to your SIEM and any other third parties concerned.
This can all be achieved securely using TLS, ensuring logs transiting the cloud are delivered with integrity and away from prying eyes.
Thanks for reading, do contact us with any questions.