pablo
|
e34a34acaf
|
Fix in throttling test so it doesn't fail around midnight.
|
2021-01-02 23:49:10 +01:00 |
|
pablo
|
def858ef6a
|
Modified input format of instructions for ParsingFlowGenerator. Previous dict wouldn't allow for more than one SecondaryFeaturesFieldInstructions class pointer.
|
2020-12-31 19:02:09 +01:00 |
|
pablo
|
2b249063e0
|
Created a new flow generator + tests for it.
|
2020-12-31 18:28:48 +01:00 |
|
pablo
|
b8d4893026
|
Mini syntax fix.
|
2020-12-31 18:14:44 +01:00 |
|
pablo
|
cb553b5f7e
|
Minor fixes in parsing utils.
|
2020-12-29 20:42:21 +01:00 |
|
pablo
|
3b79ba06d8
|
Created parsing_utils module to refactor HTML parsing and validation actions.
|
2020-12-29 17:38:17 +01:00 |
|
pablo
|
3f9a6d8e53
|
Integrated throttling in capturer.
|
2020-12-27 12:35:02 +01:00 |
|
pablo
|
d136144a4e
|
Throttling checks are now lazy.
|
2020-12-26 20:25:56 +01:00 |
|
pablo
|
2a9483981e
|
Implemented a new throttling module to remove redundance in the project.
|
2020-12-26 18:54:04 +01:00 |
|
pablo
|
f207dd5dda
|
Started integration branch.
|
2020-12-26 12:12:44 +01:00 |
|
pablo
|
0086cf2b4c
|
Improved logging in refresher.py
|
2020-11-15 13:21:08 +01:00 |
|
pablo
|
e939d67467
|
Improvements in listing page URL generation.
|
2020-11-15 12:54:17 +01:00 |
|
pablo
|
a61fac72f7
|
Typing, docstrings and formatting of explorer.py
|
2020-11-03 21:55:09 +01:00 |
|
pablo
|
f53a65834b
|
Turned method static.
|
2020-11-03 14:00:51 +01:00 |
|
pablo
|
43236c2884
|
Typing, docstrings, formatting for capturer.py
|
2020-11-03 13:50:36 +01:00 |
|
pablo
|
3cf7dd8bd9
|
Typing, docstrings, formatting for mysql_wrapper.py
|
2020-11-03 08:44:37 +01:00 |
|
pablo
|
e9ee23f852
|
Typing, docstrings, formatting for scrapping_utils.py
|
2020-11-03 07:43:21 +01:00 |
|
pablo
|
a79fc533ee
|
Formatting.
|
2020-11-03 07:29:17 +01:00 |
|
pablo
|
cd9c3b6e39
|
Some changes.
|
2020-11-03 07:26:06 +01:00 |
|
pablo
|
9e7194c8d9
|
URLAttacks now share a common session.
|
2020-11-02 13:08:37 +01:00 |
|
pablo
|
db04a67c4c
|
More testing code.
|
2020-11-02 12:51:20 +01:00 |
|
pablo
|
c337a33feb
|
More testing code.
|
2020-11-02 12:43:49 +01:00 |
|
pablo
|
81112a4cb9
|
More testing code.
|
2020-11-02 12:02:56 +01:00 |
|
pablo
|
51c4bdb347
|
Fixes. Code version for mysql.
|
2020-05-08 09:26:29 +02:00 |
|
pablo
|
596aaa1393
|
.
|
2020-05-05 11:36:28 +02:00 |
|
pablo
|
8d4c082a18
|
Format.
|
2020-04-26 15:06:04 +02:00 |
|
pablo
|
923649a099
|
Format. Random headers
|
2020-04-26 15:05:40 +02:00 |
|
pablo
|
af11a2e87f
|
Weird waiting distribution implemented
|
2020-04-26 14:54:27 +02:00 |
|
pablo
|
df032328e9
|
Formatting and todos.
|
2020-04-25 18:26:22 +02:00 |
|
pablo
|
f0fe2b9780
|
Updated headers.
|
2020-04-25 18:17:43 +02:00 |
|
pablo
|
c8ea77e99a
|
Logging en explorer.py
|
2020-03-26 11:47:12 +01:00 |
|
pablo
|
acfeeef0d1
|
Formatting
|
2020-03-26 11:38:08 +01:00 |
|
pablo
|
9c2565f5d8
|
Logging and formatting
|
2020-03-26 11:37:32 +01:00 |
|
pablo
|
cdbb6b5325
|
Added logging to geocoder.py
|
2020-03-26 11:30:09 +01:00 |
|
pablo
|
a9242b2f3a
|
Added logging config.
|
2020-03-26 11:18:14 +01:00 |
|
pablomartincalvo
|
812bb66219
|
Refactor en geocoder para evitar error con respuestas sin resultados.
|
2019-02-09 17:25:27 +01:00 |
|
pablomartincalvo
|
5ec97ad008
|
Pequeños refactorings.
|
2019-01-07 18:09:52 +01:00 |
|
pablomartincalvo
|
227f298d8b
|
Eliminados prints innecesarios del capturer.
|
2019-01-03 19:32:40 +01:00 |
|
pablomartincalvo
|
38984822a7
|
Actualizados headers de los ataques a URL por cambios en idealsita.
|
2018-12-30 19:28:05 +01:00 |
|
pablomartincalvo
|
ed32b15bc1
|
Cambios en validacion del parser
|
2018-12-30 12:06:23 +01:00 |
|
pablomartincalvo
|
98165ce8f0
|
Cambios en validacion del parser
|
2018-12-29 11:37:43 +01:00 |
|
pablomartincalvo
|
368f8a00bb
|
Merge branch 'dev'
|
2018-12-25 18:54:06 +01:00 |
|
pablomartincalvo
|
9e251783dc
|
Corregido validacion del telefono
|
2018-12-25 18:53:20 +01:00 |
|
pablomartincalvo
|
c234679a10
|
Testeado el batch de indices en dev.
|
2018-12-23 18:30:11 +01:00 |
|
pablomartincalvo
|
d71b69a611
|
Nuevos modulos para analisis
|
2018-12-21 19:17:39 +01:00 |
|
pablomartincalvo
|
965f55755a
|
Merge branch 'dev'
|
2018-12-18 20:05:27 +01:00 |
|
pablomartincalvo
|
e304069684
|
Alterado refresher para no necesitar comprobar si hay anuncios viejos.
|
2018-12-18 20:05:08 +01:00 |
|
pablomartincalvo
|
5b245c0aed
|
Merge branch 'dev'
|
2018-12-04 21:02:56 +01:00 |
|
pablomartincalvo
|
5aba6309f0
|
Transferido el espaciado entre intentos a la memoria de python en lugar
de a una comprobacion de base de datos.
Ajustado algunos tiempos.
|
2018-12-04 21:02:30 +01:00 |
|
pablomartincalvo
|
eeb8672f0d
|
Esqueleto de los cambios necesarios para añadir informacion de visitas
al sistema.
|
2018-12-02 18:53:28 +01:00 |
|