SysFlow Python API Reference

SysFlow Reader API

class sysflow.reader.FlattenedSFReader(filename, retEntities=False)

FlattenedSFReader

This class loads a raw sysflow file, and links all Entities (header, process, container, files) with the current flow or event in the file. As a result, the user does not have to manage this information. This class supports the python iterator design pattern. Example Usage:

reader = FlattenedSFReader(trace)
head = 20 # max number of records to print
for i, (objtype, header, cont, pproc, proc, files, evt, flow) in enumerate(reader):
    exe = proc.exe
    pid = proc.oid.hpid if proc else ''
    evflow = evt or flow
    tid = evflow.tid if evflow else ''
    opFlags = utils.getOpFlagsStr(evflow.opFlags) if evflow else ''
    sTime = utils.getTimeStr(evflow.ts) if evflow else ''
    eTime = utils.getTimeStr(evflow.endTs) if flow else ''
    ret = evflow.ret if evt else ''
    res1 = ''
    if objtype == ObjectTypes.FILE_FLOW or objtype == ObjectTypes.FILE_EVT:
        res1 = files[0].path
    elif objtype == ObjectTypes.NET_FLOW:
        res1 = utils.getNetFlowStr(flow)
    numBReads = evflow.numRRecvBytes if flow else ''
    numBWrites = evflow.numWSendBytes if flow else ''
    res2 = files[1].path if files and files[1] else ''
    cont = cont.id if cont else ''
    print("|{0:30}|{1:9}|{2:26}|{3:26}|{4:30}|{5:8}|{6:8}|".format(exe, opFlags, sTime, eTime, res1, numBReads, numBWrites))
    if i == head:
        break
Parameters
  • filename (str) – the name of the sysflow file to be read.

  • retEntities (bool) – If True, the reader will return entity objects by themselves as they are seen in the sysflow file. In this case, all other objects will be set to None

Iterator

Reader returns a tuple of objects in the following order:

objtype (sysflow.objtypes.ObjectTypes) The type of entity or flow returned.

header (sysflow.entity.SFHeader) The header entity of the file.

pod (sysflow.entity.Pod) The pod associated with the flow/evt, or None if no pod.

cont (sysflow.entity.Container) The container associated with the flow/evt, or None if no container.

pproc (sysflow.entity.Process) The parent process associated with the flow/evt.

proc (sysflow.entity.Process) The process associated with the flow/evt.

files (tuple of sysflow.entity.File) Any files associated with the flow/evt.

evt (sysflow.event.{ProcessEvent,FileEvent}) If the record is an event, it will be returned here. Otherwise this variable will be None. objtype will indicate the type of event.

flow (sysflow.flow.{NetworkFlow,FileFlow}) If the record is a flow, it will be returned here. Otherwise this variable will be None. objtype will indicate the type of flow.

getProcess(oid)

Returns a Process Object given a process object id.

Parameters

oid (sysflow.type.OID) – the object id of the Process Object requested

Return type

sysflow.entity.Process

Returns

the desired process object or None if no process object is available.

class sysflow.reader.NestedNamespace(**kwargs)
class sysflow.reader.SFReader(filename)

SFReader

This class loads a raw sysflow file, and returns each entity/flow one by one. It is the user’s responsibility to link the related objects together through the OID. This class supports the python iterator design pattern. Example Usage:

reader = SFReader("./sysflowfile.sf")
for name, sf in reader:
    if name == "sysflow.entity.SFHeader":
       //do something with the header object
    elif name == "sysflow.entity.Container":
       //do something with the container object
    elif name == "sysflow.entity.Process":
       //do something with the Process object
    ....
Parameters

filename (str) – the name of the sysflow file to be read.

SysFlow Formatter API

class sysflow.formatter.SFFormatter(reader, defs=[])

SFFormatter

This class takes a FlattenedSFReader, and exports SysFlow as either JSON, CSV or Pretty Print . Example Usage:

reader = FlattenedSFReader(trace, False)
formatter = SFFormatter(reader)
fields=args.fields.split(',') if args.fields else None
if args.output == 'json':
    if args.file is not None:
        formatter.toJsonFile(args.file, fields=fields)
    else:
        formatter.toJsonStdOut(fields=fields)
elif args.output == 'csv' and args.file is not None:
    formatter.toCsvFile(args.file, fields=fields)
elif args.output == 'str':
    formatter.toStdOut(fields=fields)
Parameters
applyFuncJson(func, fields=None, expr=None)

Enables a delegate function to be applied to each JSON record read.

Parameters
  • func (function) – delegate function of the form func(str)

  • fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields

  • expr (str) – a sfql filter expression

enableAllFields()

Enables all available fields to be added to the output by default.

enablePodFields()

Enables fields related to pods to be added to the output by default.

getFields()

Returns a list with available SysFlow fields and their descriptions.

toCsvFile(path, fields=None, header=True, expr=None)

Writes SysFlow to CSV file.

Parameters
  • path (str) – the full path of the output file.

  • fields (list) – a list of the SysFlow fields to be exported in the JSON. See formatter.py for a list of fields

  • expr (str) – a sfql filter expression

toDataframe(fields=None, expr=None)

Enables a delegate function to be applied to each JSON record read.

Parameters
  • func (function) – delegate function of the form func(str)

  • fields (list) – a list of the SysFlow fields to be exported in the JSON. See formatter.py for a list of fields

  • expr (str) – a sfql filter expression

toJson(fields=None, flat=False, expr=None)

Writes SysFlow as JSON object.

Parameters
  • fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields

  • expr (str) – a sfql filter expression

Flat

specifies if JSON output should be flattened

toJsonFile(path, fields=None, flat=False, expr=None)

Writes SysFlow to JSON file.

Parameters
  • path (str) – the full path of the output file.

  • fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields

  • expr (str) – a sfql filter expression

Flat

specifies if JSON output should be flattened

toJsonStdOut(fields=None, flat=False, expr=None)

Writes SysFlow as JSON to stdout.

Parameters
  • fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields

  • expr (str) – a sfql filter expression

Flat

specifies if JSON output should be flattened

toStdOut(fields=['ts_uts', 'type', 'proc.exe', 'proc.args', 'pproc.pid', 'proc.pid', 'proc.tid', 'opflags', 'res', 'flow.rbytes', 'flow.wbytes', 'container.id'], pretty_headers=True, showindex=True, expr=None)

Writes SysFlow as a tabular pretty print form to stdout.

Parameters
  • fields (list) – a list of the SysFlow fields to be exported in the JSON. See formatter.py for a list of fields

  • pretty_headers (bool) – print table headers in pretty format.

  • showindex (bool) – show record number.

  • expr (str) – a sfql filter expression

SysFlow Object Types

class sysflow.objtypes.ObjectTypes(value)

ObjectTypes

Enumeration representing each of the object types:

HEADER = 0, CONT = 1, PROC = 2, FILE = 3, PROC_EVT = 4, NET_FLOW = 5, FILE_FLOW = 6, FILE_EVT = 7 PROC_FLOW = 8 POD = 9 K8S_EVT = 10

SysFlow Utils API

sysflow.utils.getIpIntStr(ipInt)

Converts an IP address in host order integer to a string representation.

Parameters

ipInt – an IP address integer

Return type

str

Returns

A string representation of the IP address

sysflow.utils.getNetFlowStr(nf)

Converts a NetworkFlow into a string representation.

Parameters

nf (sysflow.schema_classes.SchemaClasses.sysflow.flow.NetworkFlowClass) – a NetworkFlow object.

Return type

str

Returns

A string representation of the NetworkFlow in form (sip:sport-dip:dport).

sysflow.utils.getOpFlags(opFlags)

Converts a sysflow operations flag bitmap into a set representation.

Parameters

opflag (int) – An operations bitmap from a flow or event.

Return type

set

Returns

A set representation of the operations bitmap.

sysflow.utils.getOpFlagsStr(opFlags)

Converts a sysflow operations flag bitmap into a string representation.

Parameters

opflag (int) – An operations bitmap from a flow or event.

Return type

str

Returns

A string representation of the operations bitmap.

sysflow.utils.getOpStr(opFlags)

Converts a sysflow operations into a string representation.

Parameters

opflag (int) – An operations bitmap from a flow or event.

Return type

str

Returns

A string representation of the operations bitmap.

sysflow.utils.getOpenFlags(openFlags)

Converts a sysflow open modes flag bitmap into a set representation.

Parameters

opflag – An open modes bitmap from a flow or event.

Return type

set

Returns

A set representation of the open modes bitmap.

sysflow.utils.getTimeStr(ts)

Converts a nanosecond ts into a string representation.

Parameters

ts (int) – A nanosecond epoch.

Return type

str

Returns

A string representation of the timestamp in %m/%d/%YT%H:%M:%S.%f format.

sysflow.utils.getTimeStrIso8601(ts)

Converts a nanosecond ts into a string representation in UTC time zone.

Parameters

ts (int) – A nanosecond epoch.

Return type

str

Returns

A string representation of the timestamp in ISO 8601 format.

SysFlow Class

Container Class

File Class

Header Class

Process Class

File Event

Process Event

File Flow

Network Flow