SysFlow Python API Reference¶
SysFlow Reader API¶
- class sysflow.reader.FlattenedSFReader(filename, retEntities=False)¶
FlattenedSFReader
This class loads a raw sysflow file, and links all Entities (header, process, container, files) with the current flow or event in the file. As a result, the user does not have to manage this information. This class supports the python iterator design pattern. Example Usage:
reader = FlattenedSFReader(trace) head = 20 # max number of records to print for i, (objtype, header, cont, pproc, proc, files, evt, flow) in enumerate(reader): exe = proc.exe pid = proc.oid.hpid if proc else '' evflow = evt or flow tid = evflow.tid if evflow else '' opFlags = utils.getOpFlagsStr(evflow.opFlags) if evflow else '' sTime = utils.getTimeStr(evflow.ts) if evflow else '' eTime = utils.getTimeStr(evflow.endTs) if flow else '' ret = evflow.ret if evt else '' res1 = '' if objtype == ObjectTypes.FILE_FLOW or objtype == ObjectTypes.FILE_EVT: res1 = files[0].path elif objtype == ObjectTypes.NET_FLOW: res1 = utils.getNetFlowStr(flow) numBReads = evflow.numRRecvBytes if flow else '' numBWrites = evflow.numWSendBytes if flow else '' res2 = files[1].path if files and files[1] else '' cont = cont.id if cont else '' print("|{0:30}|{1:9}|{2:26}|{3:26}|{4:30}|{5:8}|{6:8}|".format(exe, opFlags, sTime, eTime, res1, numBReads, numBWrites)) if i == head: break
- Parameters
filename (str) – the name of the sysflow file to be read.
retEntities (bool) – If True, the reader will return entity objects by themselves as they are seen in the sysflow file. In this case, all other objects will be set to None
- Iterator
Reader returns a tuple of objects in the following order:
objtype (
sysflow.objtypes.ObjectTypes
) The type of entity or flow returned.header (
sysflow.entity.SFHeader
) The header entity of the file.pod (
sysflow.entity.Pod
) The pod associated with the flow/evt, or None if no pod.cont (
sysflow.entity.Container
) The container associated with the flow/evt, or None if no container.pproc (
sysflow.entity.Process
) The parent process associated with the flow/evt.proc (
sysflow.entity.Process
) The process associated with the flow/evt.files (tuple of
sysflow.entity.File
) Any files associated with the flow/evt.evt (
sysflow.event.{ProcessEvent,FileEvent}
) If the record is an event, it will be returned here. Otherwise this variable will be None. objtype will indicate the type of event.flow (
sysflow.flow.{NetworkFlow,FileFlow}
) If the record is a flow, it will be returned here. Otherwise this variable will be None. objtype will indicate the type of flow.
- getProcess(oid)¶
Returns a Process Object given a process object id.
- Parameters
oid (sysflow.type.OID) – the object id of the Process Object requested
- Return type
sysflow.entity.Process
- Returns
the desired process object or None if no process object is available.
- class sysflow.reader.NestedNamespace(**kwargs)¶
- class sysflow.reader.SFReader(filename)¶
SFReader
This class loads a raw sysflow file, and returns each entity/flow one by one. It is the user’s responsibility to link the related objects together through the OID. This class supports the python iterator design pattern. Example Usage:
reader = SFReader("./sysflowfile.sf") for name, sf in reader: if name == "sysflow.entity.SFHeader": //do something with the header object elif name == "sysflow.entity.Container": //do something with the container object elif name == "sysflow.entity.Process": //do something with the Process object ....
- Parameters
filename (str) – the name of the sysflow file to be read.
SysFlow Formatter API¶
- class sysflow.formatter.SFFormatter(reader, defs=[])¶
SFFormatter
This class takes a FlattenedSFReader, and exports SysFlow as either JSON, CSV or Pretty Print . Example Usage:
reader = FlattenedSFReader(trace, False) formatter = SFFormatter(reader) fields=args.fields.split(',') if args.fields else None if args.output == 'json': if args.file is not None: formatter.toJsonFile(args.file, fields=fields) else: formatter.toJsonStdOut(fields=fields) elif args.output == 'csv' and args.file is not None: formatter.toCsvFile(args.file, fields=fields) elif args.output == 'str': formatter.toStdOut(fields=fields)
- Parameters
reader (sysflow.reader.FlattenedSFReader) – A reader representing the sysflow file being read.
defs (list) – A list of paths to filter definitions.
- applyFuncJson(func, fields=None, expr=None)¶
Enables a delegate function to be applied to each JSON record read.
- Parameters
func (function) – delegate function of the form func(str)
fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression
- enableAllFields()¶
Enables all available fields to be added to the output by default.
- enablePodFields()¶
Enables fields related to pods to be added to the output by default.
- getFields()¶
Returns a list with available SysFlow fields and their descriptions.
- toCsvFile(path, fields=None, header=True, expr=None)¶
Writes SysFlow to CSV file.
- Parameters
path (str) – the full path of the output file.
fields (list) – a list of the SysFlow fields to be exported in the JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression
- toDataframe(fields=None, expr=None)¶
Enables a delegate function to be applied to each JSON record read.
- Parameters
func (function) – delegate function of the form func(str)
fields (list) – a list of the SysFlow fields to be exported in the JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression
- toJson(fields=None, flat=False, expr=None)¶
Writes SysFlow as JSON object.
- Parameters
fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression
- Flat
specifies if JSON output should be flattened
- toJsonFile(path, fields=None, flat=False, expr=None)¶
Writes SysFlow to JSON file.
- Parameters
path (str) – the full path of the output file.
fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression
- Flat
specifies if JSON output should be flattened
- toJsonStdOut(fields=None, flat=False, expr=None)¶
Writes SysFlow as JSON to stdout.
- Parameters
fields (list) – a list of the SysFlow fields to be exported in JSON. See formatter.py for a list of fields
expr (str) – a sfql filter expression
- Flat
specifies if JSON output should be flattened
- toStdOut(fields=['ts_uts', 'type', 'proc.exe', 'proc.args', 'pproc.pid', 'proc.pid', 'proc.tid', 'opflags', 'res', 'flow.rbytes', 'flow.wbytes', 'container.id'], pretty_headers=True, showindex=True, expr=None)¶
Writes SysFlow as a tabular pretty print form to stdout.
- Parameters
fields (list) – a list of the SysFlow fields to be exported in the JSON. See formatter.py for a list of fields
pretty_headers (bool) – print table headers in pretty format.
showindex (bool) – show record number.
expr (str) – a sfql filter expression
SysFlow Object Types¶
- class sysflow.objtypes.ObjectTypes(value)¶
ObjectTypes
- Enumeration representing each of the object types:
HEADER = 0, CONT = 1, PROC = 2, FILE = 3, PROC_EVT = 4, NET_FLOW = 5, FILE_FLOW = 6, FILE_EVT = 7 PROC_FLOW = 8 POD = 9 K8S_EVT = 10
SysFlow Utils API¶
- sysflow.utils.getIpIntStr(ipInt)¶
Converts an IP address in host order integer to a string representation.
- Parameters
ipInt – an IP address integer
- Return type
str
- Returns
A string representation of the IP address
- sysflow.utils.getNetFlowStr(nf)¶
Converts a NetworkFlow into a string representation.
- Parameters
nf (sysflow.schema_classes.SchemaClasses.sysflow.flow.NetworkFlowClass) – a NetworkFlow object.
- Return type
str
- Returns
A string representation of the NetworkFlow in form (sip:sport-dip:dport).
- sysflow.utils.getOpFlags(opFlags)¶
Converts a sysflow operations flag bitmap into a set representation.
- Parameters
opflag (int) – An operations bitmap from a flow or event.
- Return type
set
- Returns
A set representation of the operations bitmap.
- sysflow.utils.getOpFlagsStr(opFlags)¶
Converts a sysflow operations flag bitmap into a string representation.
- Parameters
opflag (int) – An operations bitmap from a flow or event.
- Return type
str
- Returns
A string representation of the operations bitmap.
- sysflow.utils.getOpStr(opFlags)¶
Converts a sysflow operations into a string representation.
- Parameters
opflag (int) – An operations bitmap from a flow or event.
- Return type
str
- Returns
A string representation of the operations bitmap.
- sysflow.utils.getOpenFlags(openFlags)¶
Converts a sysflow open modes flag bitmap into a set representation.
- Parameters
opflag – An open modes bitmap from a flow or event.
- Return type
set
- Returns
A set representation of the open modes bitmap.
- sysflow.utils.getTimeStr(ts)¶
Converts a nanosecond ts into a string representation.
- Parameters
ts (int) – A nanosecond epoch.
- Return type
str
- Returns
A string representation of the timestamp in %m/%d/%YT%H:%M:%S.%f format.
- sysflow.utils.getTimeStrIso8601(ts)¶
Converts a nanosecond ts into a string representation in UTC time zone.
- Parameters
ts (int) – A nanosecond epoch.
- Return type
str
- Returns
A string representation of the timestamp in ISO 8601 format.