intelmq.bots.parsers.shadowserver package

Submodules

intelmq.bots.parsers.shadowserver.parser module

Copyright (C) 2016 by Bundesamt für Sicherheit in der Informationstechnik Software engineering by Intevation GmbH

This is an “all-in-one” parser for a lot of shadowserver feeds. It depends on the configuration in the file “config.py” which holds information on how to treat certain shadowserverfeeds. It uses the report field extra.file_name to determine which config should apply, so this field is required.

This parser will only work with csv files named like 2019-01-01-scan_http-country-geo.csv.

Optional parameters:
overwrite: Bool, default False. If True, it keeps the report’s

feed.name and does not override it with the corresponding feed name.

feedname: The fixed feed name to use if it should not automatically detected.

intelmq.bots.parsers.shadowserver.parser.BOT

alias of intelmq.bots.parsers.shadowserver.parser.ShadowserverParserBot

class intelmq.bots.parsers.shadowserver.parser.ShadowserverParserBot(bot_id: str, start: bool = False, sighup_event=None, disable_multithreading: bool = None)

Bases: intelmq.lib.bot.ParserBot

Parse all ShadowServer feeds

feedname = None
init()
overwrite = False
parse(report)

A generator yielding the single elements of the data.

Comments, headers etc. can be processed here. Data needed by self.parse_line can be saved in self.tempdata (list).

Default parser yields stripped lines. Override for your use or use an existing parser, e.g.:

parse = ParserBot.parse_csv
You should do that for recovering lines too.

recover_line = ParserBot.recover_line_csv

parse_line(row, report)

A generator which can yield one or more messages contained in line.

Report has the full message, thus you can access some metadata. Override for your use.

recover_line(line: Union[dict, str, None] = None) → str

Converts dictionaries to csv. self.csv_fieldnames must be list of fields.

intelmq.bots.parsers.shadowserver.parser_json module

Shadowserver JSON Parser

SPDX-FileCopyrightText: 2020 Intelmq Team <intelmq-team@cert.at> SPDX-License-Identifier: AGPL-3.0-or-later

intelmq.bots.parsers.shadowserver.parser_json.BOT

alias of intelmq.bots.parsers.shadowserver.parser_json.ShadowserverJSONParserBot

class intelmq.bots.parsers.shadowserver.parser_json.ShadowserverJSONParserBot(bot_id: str, start: bool = False, sighup_event=None, disable_multithreading: bool = None)

Bases: intelmq.lib.bot.ParserBot

Parse all Shadowserver feeds in JSON format (data coming from the reports API) Shadowserver JSON Parser

Parameters

feedname (str) – The name of the feed

feedname = None
get_value_from_config(data, entry)

Given a specific config, get the value for that data based on the entry

init()
overwrite = True
parse(report)

A generator yielding the single elements of the data.

Comments, headers etc. can be processed here. Data needed by self.parse_line can be saved in self.tempdata (list).

Default parser yields stripped lines. Override for your use or use an existing parser, e.g.:

parse = ParserBot.parse_csv
You should do that for recovering lines too.

recover_line = ParserBot.recover_line_csv

parse_line(line: Any, report: intelmq.lib.message.Report)

A generator which can yield one or more messages contained in line.

Report has the full message, thus you can access some metadata. Override for your use.

recover_line(line: dict) → str

Reverse of parse for JSON pulses.

Recovers a fully functional report with only the problematic pulse. Using a string as input here is not possible, as the input may span over multiple lines. Output is not identical to the input, but has the same content.

Parameters

line as dict. (The) –

Returns

The JSON-encoded line as string.

Return type

str

Module contents