SysFlow Python API Reference¶

SysFlow Reader API¶

class sysflow.reader.FlattenedSFReader(filename, retEntities=False)¶

FlattenedSFReader

This class loads a raw sysflow file, and links all Entities (header, process, container, files) with the current flow or event in the file. As a result, the user does not have to manage this information. This class supports the python iterator design pattern. Example Usage:

reader = FlattenedSFReader(trace)
head = 20 # max number of records to print
for i, (objtype, header, cont, pproc, proc, files, evt, flow) in enumerate(reader):
    exe = proc.exe
    pid = proc.oid.hpid if proc else ''
    evflow = evt or flow
    tid = evflow.tid if evflow else ''
    opFlags = utils.getOpFlagsStr(evflow.opFlags) if evflow else ''
    sTime = utils.getTimeStr(evflow.ts) if evflow else ''
    eTime = utils.getTimeStr(evflow.endTs) if flow else ''
    ret = evflow.ret if evt else ''
    res1 = ''
    if objtype == ObjectTypes.FILE_FLOW or objtype == ObjectTypes.FILE_EVT:
        res1 = files[0].path
    elif objtype == ObjectTypes.NET_FLOW:
        res1 = utils.getNetFlowStr(flow)
    numBReads = evflow.numRRecvBytes if flow else ''
    numBWrites = evflow.numWSendBytes if flow else ''
    res2 = files[1].path if files and files[1] else ''
    cont = cont.id if cont else ''
    print("|{0:30}|{1:9}|{2:26}|{3:26}|{4:30}|{5:8}|{6:8}|".format(exe, opFlags, sTime, eTime, res1, numBReads, numBWrites))
    if i == head:
        break

Parameters

filename (str) – the name of the sysflow file to be read.
retEntities (bool) – If True, the reader will return entity objects by themselves as they are seen in the sysflow file. In this case, all other objects will be set to None

Iterator

Reader returns a tuple of objects in the following order:

objtype (sysflow.objtypes.ObjectTypes) The type of entity or flow returned.

header (sysflow.entity.SFHeader) The header entity of the file.

pod (sysflow.entity.Pod) The pod associated with the flow/evt, or None if no pod.

cont (sysflow.entity.Container) The container associated with the flow/evt, or None if no container.

pproc (sysflow.entity.Process) The parent process associated with the flow/evt.

proc (sysflow.entity.Process) The process associated with the flow/evt.

files (tuple of sysflow.entity.File) Any files associated with the flow/evt.

evt (sysflow.event.{ProcessEvent,FileEvent}) If the record is an event, it will be returned here. Otherwise this variable will be None. objtype will indicate the type of event.

flow (sysflow.flow.{NetworkFlow,FileFlow}) If the record is a flow, it will be returned here. Otherwise this variable will be None. objtype will indicate the type of flow.

getProcess(oid)¶

Returns a Process Object given a process object id.

Parameters: oid (sysflow.type.OID) – the object id of the Process Object requested
Return type: sysflow.entity.Process
Returns: the desired process object or None if no process object is available.

class sysflow.reader.NestedNamespace(**kwargs)¶

class sysflow.reader.SFReader(filename)¶

SFReader

This class loads a raw sysflow file, and returns each entity/flow one by one. It is the user’s responsibility to link the related objects together through the OID. This class supports the python iterator design pattern. Example Usage:

reader = SFReader("./sysflowfile.sf")
for name, sf in reader:
    if name == "sysflow.entity.SFHeader":
       //do something with the header object
    elif name == "sysflow.entity.Container":
       //do something with the container object
    elif name == "sysflow.entity.Process":
       //do something with the Process object
    ....

Parameters: filename (str) – the name of the sysflow file to be read.

SysFlow Formatter API¶

class sysflow.formatter.SFFormatter(reader, defs=[])¶

SFFormatter

This class takes a FlattenedSFReader, and exports SysFlow as either JSON, CSV or Pretty Print . Example Usage:

reader = FlattenedSFReader(trace, False)
formatter = SFFormatter(reader)
fields=args.fields.split(',') if args.fields else None
if args.output == 'json':
    if args.file is not None:
        formatter.toJsonFile(args.file, fields=fields)
    else:
        formatter.toJsonStdOut(fields=fields)
elif args.output == 'csv' and args.file is not None:
    formatter.toCsvFile(args.file, fields=fields)
elif args.output == 'str':
    formatter.toStdOut(fields=fields)

Parameters

reader (sysflow.reader.FlattenedSFReader) – A reader representing the sysflow file being read.
defs (list) – A list of paths to filter definitions.

applyFuncJson(func, fields=None, expr=None)¶

Enables a delegate function to be applied to each JSON record read.

Parameters

func (function) – delegate function of the form func(str)
fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression

enableAllFields()¶: Enables all available fields to be added to the output by default.

enablePodFields()¶: Enables fields related to pods to be added to the output by default.

getFields()¶: Returns a list with available SysFlow fields and their descriptions.

toCsvFile(path, fields=None, header=True, expr=None)¶

Writes SysFlow to CSV file.

Parameters

path (str) – the full path of the output file.
fields (list) – a list of the SysFlow fields to be exported in the JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression

toDataframe(fields=None, expr=None)¶

Enables a delegate function to be applied to each JSON record read.

Parameters

func (function) – delegate function of the form func(str)
fields (list) – a list of the SysFlow fields to be exported in the JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression

toJson(fields=None, flat=False, expr=None)¶

Writes SysFlow as JSON object.

Parameters

fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression

Flat

specifies if JSON output should be flattened

toJsonFile(path, fields=None, flat=False, expr=None)¶

Writes SysFlow to JSON file.

Parameters

path (str) – the full path of the output file.
fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression

Flat

specifies if JSON output should be flattened

toJsonStdOut(fields=None, flat=False, expr=None)¶

Writes SysFlow as JSON to stdout.

Parameters

fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression

Flat

specifies if JSON output should be flattened

toStdOut(fields=['ts_uts', 'type', 'proc.exe', 'proc.args', 'pproc.pid', 'proc.pid', 'proc.tid', 'opflags', 'res', 'flow.rbytes', 'flow.wbytes', 'container.id'], pretty_headers=True, showindex=True, expr=None)¶

Writes SysFlow as a tabular pretty print form to stdout.

Parameters

fields (list) – a list of the SysFlow fields to be exported in the JSON. See formatter.py for a list of fields
pretty_headers (bool) – print table headers in pretty format.
showindex (bool) – show record number.
expr (str) – a sfql filter expression

SysFlow Object Types¶

class sysflow.objtypes.ObjectTypes(value)¶

ObjectTypes

Enumeration representing each of the object types:: HEADER = 0, CONT = 1, PROC = 2, FILE = 3, PROC_EVT = 4, NET_FLOW = 5, FILE_FLOW = 6, FILE_EVT = 7 PROC_FLOW = 8 POD = 9 K8S_EVT = 10

SysFlow Utils API¶

sysflow.utils.getIpIntStr(ipInt)¶

Converts an IP address in host order integer to a string representation.

Parameters: ipInt – an IP address integer
Return type: str
Returns: A string representation of the IP address

sysflow.utils.getNetFlowStr(nf)¶

Converts a NetworkFlow into a string representation.

Parameters: nf (sysflow.schema_classes.SchemaClasses.sysflow.flow.NetworkFlowClass) – a NetworkFlow object.
Return type: str
Returns: A string representation of the NetworkFlow in form (sip:sport-dip:dport).

sysflow.utils.getOpFlags(opFlags)¶

Converts a sysflow operations flag bitmap into a set representation.

Parameters: opflag (int) – An operations bitmap from a flow or event.
Return type: set
Returns: A set representation of the operations bitmap.

sysflow.utils.getOpFlagsStr(opFlags)¶

Converts a sysflow operations flag bitmap into a string representation.

Parameters: opflag (int) – An operations bitmap from a flow or event.
Return type: str
Returns: A string representation of the operations bitmap.

sysflow.utils.getOpStr(opFlags)¶

Converts a sysflow operations into a string representation.

Parameters: opflag (int) – An operations bitmap from a flow or event.
Return type: str
Returns: A string representation of the operations bitmap.

sysflow.utils.getOpenFlags(openFlags)¶

Converts a sysflow open modes flag bitmap into a set representation.

Parameters: opflag – An open modes bitmap from a flow or event.
Return type: set
Returns: A set representation of the open modes bitmap.

sysflow.utils.getTimeStr(ts)¶

Converts a nanosecond ts into a string representation.

Parameters: ts (int) – A nanosecond epoch.
Return type: str
Returns: A string representation of the timestamp in %m/%d/%YT%H:%M:%S.%f format.

SysFlow Python API Reference¶

SysFlow Reader API¶

SysFlow Formatter API¶

SysFlow Object Types¶

SysFlow Utils API¶

SysFlow Class¶

Container Class¶

File Class¶

Header Class¶

Process Class¶

File Event¶

Process Event¶

File Flow¶

Network Flow¶