api_external module

The api_external module is part of the nmeta suite, but is run separately

This module runs a class and methods for an API that exposes an interface into nmeta MongoDB collections.

It leverages the Eve Python REST API Framework

class api_external.ExternalAPI(config)

Bases: baseclass.BaseClass

This class provides methods for the External API

class FlowUI

Bases: object

An object that represents a flow record to be sent in response to the WebUI. Features:

• Flow direction normalised to direction of first packet in flow
• Src and Dst are IP or Layer 2 to optimise screen space
• Extra data included for hover-over tips

Note that there should not be any display-specific data (i.e. don’t send any HTML, leave this to the client code)

response()

Return a dictionary object of flow parameters for sending in response

run()

Run the External API instance

Note that API definitions are from previously imported files from api_definitions subdirectory

response_pi_rate(items)

Update the response with the packet_in rate. Hooked from on_fetched_resource_pi_rate

Returns key/values for packet-in processing time in API response: - timestamp - pi_rate

response_pi_time(items)

Update the response with the packet_time min, avg and max. Hooked from on_fetched_resource_pi_time

Returns key/values for packet-in processing time in API response: - timestamp - ryu_time_max - ryu_time_min - ryu_time_avg - ryu_time_period - ryu_time_records - pi_time_max - pi_time_min - pi_time_avg - pi_time_period - pi_time_records

If no data found within time period then returns without key/values

response_controller_summary(items)

Update the response with the packet_in rate, packet processing time stats

Hooked from on_fetched_resource_controller_summary

Rounds seconds results

response_identities_ui(items)

Populate the response with identities that are filtered:

• Reverse sort by harvest time
• Deduplicate by id_hash, only returning most recent per id_hash
• Includes possibly stale records
• Check DNS A records to see if they are from a CNAME

Hooked from on_fetched_resource_<name>

response_flows_removed_stats_count(items)

Return count of removed flows collection

response_flows_removed_src_bytes_sent(items)

Returns removed flow bytes sent by session source IP (deduplicated for flows crossing multiple switches), enriched with identity metadata.

response_flows_removed_src_bytes_received(items)

Returns removed flow bytes received by session source IP (deduplicated for flows crossing multiple switches), enriched with identity metadata.

response_flows_removed_dst_bytes_sent(items)

Returns removed flow bytes sent by session destination IP (deduplicated for flows crossing multiple switches), enriched with identity metadata.

response_flows_removed_dst_bytes_received(items)

Returns removed flow bytes received by session destination IP (dedup for flows crossing multiple switches), enriched with identity metadata.

response_flows_ui(items)

Populate the response with flow entries that are filtered:

• Reverse sort by initial ingest time
• Deduplicate by flow_hash, only returning most recent per flow_hash
• Enrich with TBD

Hooked from on_fetched_resource_<name>

response_switches_count(items)

Populate the response with number of connected switches.

flow_match(flow, flows_filterlogicselector, flows_filtertypeselector, filter_string)

Passed an instance of FlowUI class, a logic selector, filter type and filter string.

Return a boolean on whether or not that theres a match.

flow_augment_record(record)

Passed a record of a single flow from the packet_ins database collection.

Create FlowUI class instance, add in known data and augment with identity data. Logic is specific to the webUI user experience.

Return the FlowUI class instance

get_flow_data_xfer(record)

Passed a record of a single flow from the packet_ins database collection.

Enrich this by looking up data transfer stats (which may not exist) in flow_rems database collection, and return dictionary of the values.

Note that the data sent (tx) and received (rx) records will have different flow hashes.

get_classification(flow_hash)

Passed flow_hash and return a dictionary of a classification object for the flow_hash (if found), otherwise a dictionary of an empty classification object.

flow_normalise_direction(record)

Passed a dictionary of an flow record and return a similar dictionary that has sources and destinations normalised to the direction of the first observed packet in the flow

get_flow_client_ip(flow_hash)

Find the IP that is the originator of a flow searching forward by flow_hash

Finds first packet seen for the flow_hash within the time limit and returns the source IP, otherwise 0,

get_id(ip_addr)

Passed an IP address. Look this up for matching identity metadata and return a string that contains either the original IP address or an identity string

get_dns_ip(service_name)

Use this to get an IP address for a DNS lookup that returned a CNAME Passed a DNS CNAME and look this up in identities collection to see if there is a DNS A record, and if so return the IP address, otherwise return an empty string.

get_pi_rate(test=0)

Calculate packet-in rate by querying packet_ins database collection.

Setting test=1 returns database query execution statistics

get_pi_time()

Calculate packet processing time statistics by querying the pi_time database collection.

api_external.enumerate_eth_type(eth_type)

Passed an eth_type (in decimal) and return an enumerated version, or if not found, return the original value. Example, pass this function value 2054 and it return will be ‘ARP’

api_external.hovertext_eth_type(eth_type)

Passed an eth_type (decimal, not enumerated) and return it wrapped in extra text to convey context

api_external.enumerate_ip_proto(ip_proto)

Passed an IP protocol number (in decimal) and return an enumerated version, or if not found, return the original value. Example, pass this function value 6 and it return will be ‘TCP’

api_external.hovertext_ip_proto(ip_proto)

Passed an IP protocol number (decimal, not enumerated) and return it wrapped in extra text to convey context

api_external.hovertext_ip_addr(ip_addr)

Passed an IP address and return it wrapped in extra text to convey context