Time to time our customers need to group some information available in the internet or other sources, or gather and group it again and again to track changes. We have experience in building lightweight crawlers, including those, that have to interact with complicated online applications, to get necessary information, and doing analysis of gathered information.
Depending on amount of information we can suggest different approaches for storing and analysing of gathered information, using cloud services for distributed databases and MapRed processing in particular cases. We provide API to access to gathered and processed information if neccessary.
Depending on requirements, processing and delivering of results can be either delayed or realtime or mixed (lambda architecture). In all cases we handle integration with our customers’ applications.