python - spider_closed function is not fired -

-

i want log data when spider finishes working.

i make function inside spider.

def spider_closed(self, spider):         log.msg('the spider {0} finished working on'.format(self.name), level=log.info)

but function not getting fired.

i using scrapy 0.20 python 2.7

please note can log log , logging lot of data can see in log file. logging working me.

edit

i guess have dispatch signal signals.spider_closed don't know how that.

you should attach method receiver signalmanager signal dispatcher:

from scrapy.xlib.pydispatch import dispatcher scrapy import signals  class myspider(spider):     def __init__(self, *args, **kwargs):         dispatcher.connect(self.spider_closed, signal=signals.spider_closed)         super(myspider, self).__init__(*args, **kwargs)      def spider_closed(self, spider):         log.msg('the spider {0} finished working on'.format(self.name), level=log.info)

also see relevant threads:


Comments

Post a Comment

Popular posts from this blog

c++ - How to add Crypto++ library to Qt project -

-
Image
i downloaded crypto++ source , compiled cryptlib project in visual studio 2013, , added generated .lib file qt project, made .pro file this: qt += core gui qt += sql greaterthan(qt_major_version, 4):qt += widgets target = untitled template = app sources += main.cpp\ mainwindow.cpp headers += mainwindow.h \ databasecontrol.h \ test.h forms += mainwindow.ui win32:config(release, debug|release): libs += -l$$pwd/ -lcryptlib else:win32:config(debug, debug|release): libs += -l$$pwd/ -lcryptlibd else:unix: libs += -l$$pwd/ -lcryptlib includepath += $$pwd/ dependpath += $$pwd/ win32-g++:config(release, debug|release): pre_targetdeps += $$pwd/libcryptlib.a else:win32-g++:config(debug, debug|release): pre_targetdeps += $$pwd/libcryptlibd.a else:win32:!win32-g++:config(release, debug|release): pre_targetdeps += $$pwd/cryptlib.lib else:win32:!win32-g++:config(debug, debug|release): pre_targetdeps += $$pwd/cryptlibd.lib else:unix: pre_targetdeps += $$pw
Read more

jQuery Mobile app not scrolling in Firefox -

-
i have mobile app built in jquery mobile. scrolling fine in chrome & safari, fails scroll in firefox on desktop. have other instances of same app, different features, , works across browsers. life of me, cannot figure this, please help! located here: http://re-brand.us/dsus/index.html seems bug in firefox: bug log . able scroll using keyboard not trackpad on mac. are using mac? i'll keep looking see if can find way around it.
Read more

How to use vim as editor in Matlab GUI -

-
i using matlab r2013b. using gui, command windows , editor in split screen. in matlab preferences > editor/debugger > editor. there option set custom editor instead of matlab editor. set local editor /usr/bin/vim , not able open files @ anymore. there possibility use vim editor in split screen mode? i aware of !vim file.m , not solution looking for, works satisfactory in -nodisplay mode. rather want keep split screen mode. i did not find solution 100% satisfying, using tmux intermediate solution. allows 2 use split screen 2 terminals. in 1 of can open matlab -nodesktop in terminal mode, , in other 1 vim . possible switch between both split screens. you can use functionality of gui matlab typing things commandhistory , who et cetera.
Read more