Long Term Plan (Archive)

Here is a detailed list of requirements, priorities and future plans

Priorities are rated as follows:

(1) must be in first phase to have something working
(2) necessary to have a system having more benefits than existing systems after prio 1 items, the prio items needs to be re-prioritized
(3) nice to have or giving the more benefit on the already existing and usable system

Overall Goals

The goal is to detect a DDoS on the network and be able to take steps necessary for using a mitigation device or scrubbing center. Detection is not always possible in a fully automatic way. Manual analyses and intervention must be possible.

Detection in such a way that a DDoS attack can be detected before it actually results in a denial of service
Mitigation by announcing BGP routes

Areas of use:

Standalone detection and analyses system Detection and analyses of own network Ready to be able to mitigate by controlling a mitigation device
Shared detection and analyses system Same as standalone system, but with multi-tenant capabilities Based on flow information of the customer
NaWas management and control Determine a DDoS for a specific customer Report on running attack for a customer Report on passed attacks for a customer Multi-tenant Based on the flow information of the NaWas network itself

Inputs

Network Traffic input:

(1) flow based
- (1) netflow
- (3) Juniper: cflow / jflow
- (3) others: NetStream / IPFIX / openflow / sflow
(2) snmp statistics
(3) wiretap
- (1) stored in pcap format
- (1) inline pcap
- SPAN/RSPAN/ERSPAN
- VACL Capture
- Router IP Traffic Export (RITE)
- Embedded Packet Capture (EPC)

General Goals:

(2) flow information from multiple sources
(2) Differentiation should be made as:
- input device(s) - flow info before mitigation
- output device(s) - clean traffic
- internal devices - management, control, internal traffic mitigation devices

Detection

General Goals:

(1) analyses of network traffic to be able to determine a DDoS
(2) AS centric approach: relate IP addresses to ASN's of defined zones => internal traffic; otherwise external traffic
(1) Detecting anomalous network behaviour, either on purpose (DDoS) or not
(2) Possibility to detect an anomaly as fast as possible (about 1 second), either automatically or manual
(2) templates for thresholds and types of detection rules (hosting provider, enterprise, mail provider)
(3) reputation and fingerprint database connection from central repository (on internet or in own domain)
- default database on shipment of software

Detection methods

A. tunable thresholds for specific types of network (i.e. UDP, ICMP, pps, Mbps)

(2) pre-populated adjustable ruleset in software, based on experience
(3) updated by software upgrade
(2) icmp information incorporated, like icmp unreachable

B. baseline of traffic, learning possibilities

(2) normal traffic for past x-days
(2) increase/decrease possible on configurable percentage of change in traffic rates
(3) take into account: moment of the day, day of week, day in month, season

C. (2) signature bases database of known bad traffic (fingerprinting)

(3) per signature:
- only alerting or alerting and mitigation
- thresholds per signature

D. (3) Reputation database

IP reputation database
AS reputation database
trigger on incoming traffic from bad IP's or AS's with threshold
trigger on outgoing traffic to bad IP's or AS's with threshold

E. (3) Geo information added to external IP addresses

Output

Reporting tools are necessary for manual analyses and intervention and management reporting.

(1) Output in network graphs, top lists and triggers when anomaly occurs
(1) output for statistics and trend analyses
(1) output for signaling of anomalies (snmp traps, sms)
(2) display flow data as-is
(3) store pcap for a configurable amount of time
(1) response to commands must be "immediately". On longer actions a message should appear, like "please wait"
(3) open query language with filtering such as tcpdump ** top-x ** timeframe (i.e. "within 2014-09-01 10:00 and 2014-09-01 12:00") ** filter, alike tcpdump i.e. "select service-top-10 within 2014-09-01 10:00 and 2014-09-01 12:00 filter ip and dst host 1.2.3.4"
(3) heavy queries make take longer, but an indication should be given
(3) user abort of queries must be possible

top-10 lists

For fast interpretation of network traffic for a given timeframe.

(1) protocol used (ip, udp, etc)
(1) service used (http, ntp, dns), with or without filtering on protocol
(1) most used IP address
(3) most used IP prefixes (CIDR); filter based on given prefix length (like /24)
(2) top-10 traffic pattern; based on combination of protocol, service and IP-address
(3) add direction to top-10 traffic pattern:
- directed to internal address = inbound DDoS
- directed to external address = part of outbound DoS
(1) top-10 in CLI
(2) top-10 in webinterface
(3) button to mitigate on top-10 list (in webinterface)
(2) top-x list should give output in a reasonable amount of time (3 seconds)

Control

The web and CLI interfaces for controlling the system.

(1) CLI at first
(2)(3) Webinterface - config (3); analysing (2); reporting (3)
- (2) webinterface must be with SSL/TLS
- (1) using open standards for cross-platform modern web browsers
(1) basic tasks must be able to run on mobile devices (can be separate development by using API)
(1) API for use in existing software of own developments
(2) multi-user ** users ** groups ** activities based on group- or user rights (read, write, modify, delete)
(3) multi-tenant multiple users with their own flow sources and flow information
(2) external traffic: relate to external links, like peering and transit links
(2) internal traffic ** (2) possibility to create zones/customers(in case of NaWaS) / how-you-like-to-name-it ** (2) relation of IP prefixes to zones ("internal" traffic) ** (3) relation of ASN to zones
(3) automatic or manually activated upgrade
(3) backup and restore of configuration
(1) configuration in text file and editable with and external editor
(1) configuration must be in one place and portible

Project Management Requirements

(1) setting up development environment
(1) software will be open source
(1) re-use existing software can be used where possible (like nfsen, pcap, openflow)
(1) license based on re-used software
(1) object oriented
(1) API interface from the start
(2) CLI commands and API interface are similar
(1) secure code - use framework/guidelines for this
- necessary for code on interfacing level
- advised for other code
(3) basic code review of re-used software, such as: procedures developers have for handling vulnerabilities, quick scan of quality of code
(1) development with the different areas of use in mind
(2) flow information in a database
(1) software should run on unix based systems (linux, bsd)
(1) preferred is open platform, linux, freebsd, mac osx
(1) support for both IP6 and IP4 from the start
(2) all changes in configuration must be reported (audit chain)
(3) forensic readyness:
export / hashes
chain of custody of data
audit of changes
(2) improved packaging, perhaps Docker

Target Environment

(1) define hardware requirements
(1) software must run be able to run on dedicated hardware and on a virtual machine
(3) system must be able to handle flow data on 1Gbps line speed
(3) system must be able to handle wiretap data at 10Gbps line speed
(2) software must use separate network interfaces for flow data/wiretap (analyses interface) and management
(2) system manager should be able to configure the same network interface for analyses interface and management interface (is discouraged, but possible)
(2) access to system must be secured (ssh, VPN)
(2) interexchange of data (flow, wiretap) must be local or secured with encryption
(3) appliance must be able to be interconnected in the future, to be able to create a sensor network

Mitigation

Eventually, the DDoS detection should control a traffic scrubbing and mitigation engine.

Mitigation module preliminary design:

config: device
input: DDoS begin/end events
task: establishes mitigations
state: current-mitigations

Mitigation engine types:

Local Scrubbing engine
Openflow configuration engine
netconf router config engine

Internal Design Goals

keep time as an abstract concept to allow replaying of historical attacks just like they were happening

Future Development Ideas

flow sensor network

Provide feedback

Saved searches

Use saved searches to filter your results more quickly