Refactorizado Explorer para que el Explorer, y no el exploring_task, se encargue de postear las tareas de captura.

Creado una funcion independiente de creacion de capturas para que sea compartida entre todos aquellos servicios que las creen.
This commit is contained in:
pablomartincalvo 2018-09-21 18:19:33 +02:00
parent b77a4752b8
commit 1f372b85b6
4 changed files with 286 additions and 94 deletions

0
capturer/__init__.py Normal file
View file

23
capturer/capturer.py Normal file
View file

@ -0,0 +1,23 @@
import uuid
ads_root = 'https://www.idealista.com/inmueble/'
def create_capturing_task(referencia, db_wrapper, uuid_exploring=None):
query_parameters = {'url': ads_root + referencia,
'uuid': str(uuid.uuid4()),
'status': 'Pending'}
if uuid_exploring is None:
query_statement = """INSERT INTO capturing_tasks_logs
(uuid, write_time, status, url)
VALUES (%(uuid)s, NOW(), %(status)s, url)"""
else:
query_statement = """INSERT INTO capturing_tasks_logs
(uuid, write_time, status, url, fk_uuid_exploring)
VALUES (%(uuid)s, NOW(), %(status)s, url, %(uuid_exploring))s"""
query_parameters['uuid_exploring'] = uuid_exploring
db_wrapper.query(query_statement, query_parameters)