python webkit scrapy

ubuntu apt-get 安装pythonwebkit,jswebkit {{{ apt-get install python-webkit jswebkit }}} debian {{{ apt-get install python-jswebkit python-webkit }}} 在scrapy的settings.py中加入: {{{#!highlight python

which spider should use WEBKIT

WEBKIT_DOWNLOADER=['jxydt'] DOWNLOADER_MIDDLEWARES = { 'jx.dowloader.WebkitDownloader': 543, }

import os os.environ["DISPLAY"] = ":0" }}} dowloader.py {{{#!highlight python

!/usr/bin/env python

-- coding: utf-8 --

from scrapy.http import Request ...

more ...

Describe linux/sphinx写书 here.

{{{ pip install sphinx }}}

生成项目 {{{ sphinx-quickstart }}}

生成pdf {{{ apt-get install texlive-xetex # xelatex在此包中,先不安装下面的texlive试试 apt-get install latex-make

apt-get install texlive # 先进行下面步骤,如果出错,安装texlive make latex cd builc/latex xelatex [abc].tex #abc为tex文件名 }}}

中文内容支持 {{{ fc-list :lang=zh #查看可用中文字体 l@x201i:~/tmp/abd$ fc-list :lang=zh /usr/share/fonts/truetype/wqy/wqy-zenhei.ttc: WenQuanYi Zen ...

more ...

mysql中增加远程表federated支持

Date 二 29 十月 2013 By liugehao Category db.

在my.cnf中[mysqld]下增加一行

{{{ federated }}} 重启mysql

{{{#!highlight sql CREATE TABLE federated_table (

id int(20) NOT NULL auto_increment,

name varchar(32) NOT NULL default '',

other int(20) NOT NULL default '0',

PRIMARY KEY (id),

KEY name (name),

KEY other_key (other)

)

ENGINE=FEDERATED

DEFAULT CHARSET=latin1

CONNECTION='mysql://root:password@remote_host:9306 ...

more ...

Describe linux/SonarQube install on rhel5.7 here.

用linux自带jdk出错 安装sun(oracle)jdk [[linux/rehl5安装jdk]]

== 下载安装软件 == {{{ wget http://dist.sonar.codehaus.org/sonar-3.7.2.zip unzip sonar-3.7.2.zip mv sonar-3.7.2 /etc/sonar /etc/sonar/bin/linux-x86-64/sonar.sh console //start sonar{console | start |stop |restart|status|dump} }}} 打开浏览器,输入 http://localhost:9000/ 可见软件界面安装完毕 (default ...

more ...

在这里详述 ruby/custom_require.rb:36:in `require': cannot load such file -- mkmf (LoadError)。

{{{ root@x201i:/home/l# gem install classifier Building native extensions. This could take a while... ERROR: Error installing classifier: ERROR: Failed to build gem native extension.

    /usr/bin/ruby1.9.1 extconf.rb

/usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in require': cannot load such file -- mkmf ...

more ...

Describe linux/zeromq install on rhel6.4 with php extension here.

=== zeromq install on rhel 6.4 x64 === {{{ cd /dev/yum.repo.d/ wget http://download.opensuse.org/repositories/home:/fengshuo:/zeromq/CentOS_CentOS-6/home:fengshuo:zeromq.repo yum update yum install zeromq yum install zeromq-devel # for development }}}

=== php zermoq extension === {{{ yum install libxml2-devel

wget http://cn2.php.net/distributions/php-5.5 ...

more ...

Describe liugehao-工作-201309 here.

i. js文件压缩测试,去除注释 100% i. html css 图片压缩方法测试 i. rabbitmq 安装配置及文档编写 i. rabbitmq 与php连接测试程序编写 i. rabbitmq 性能测试 i. zeromq 安装与配置 i. zeromq php 连接测试 30%

more ...

Describe linux/rabbitmq 安装 php扩展 here.

{{{

git clone https://github.com/alanxz/rabbitmq-c cd rabbitmq-c/ git submodule init git submodule update

yum install cmake

mkdir build && cd build cmake .. cd .. && make && make install

wget http://pecl.php.net/get/amqp-1.2.0.tgz tar zxf amqp-1.2.0.tgz cd amqp-1.2.0 phpize ./configure make ...

more ...

Describe linux/rhel 5.7 安装 rabbitmq here.

=== 先安装erlang ===

{{{ yum list|grep curses rpm -ivh ncurses-devel-5.5-24.20060715.x86_64.rpm

wget http://www.erlang.org/download/otp_src_R16B01.tar.gz tar zxf otp_src_R16B01.tar.gz cd otp_src_R16B01 ./configure make && make install

}}} === 安装rabbitmq之前,设置环境变量 === {{{ export TARGET_DIR=/opt/rabbitmq export SBIN_DIR=/opt/rabbitmq/sbin export MAN_DIR=/opt/rabbitmq/man }}}

=== 安装rabbitmq === {{{ rpm ...

more ...

Describe python/virtualenv pip 技巧 here.

== virtualenv == 这里是导言吗? 用过Python的同学,肯定会对Python及程序的版本之间经常更换的api感到痛苦不以。就拿我折腾的Django来说吧,公司服务器上跑的是Django1.3、同事也是用1.3开发,但是因为我是新来,一个 pip install django 下去,就是1.4.2。好了,你自己写的Django Project自然没有问题,自己本地测试也没有问题。但是要和其他人交流的时候就蛋疼了,因为你的1.4.2跑不了1.3的程序……当然,这时,你可以选择卸载自己本地的Django,换成1.3,等你要重新测试自己的Django,怎么样,扯着蛋了吧。为了解决以上问题,virtualenv横空出世了。 正文 为了解决以上蛋疼问题,我们需要安装virtualenv。 {{{ sudo pip install virtualenv }}} 安装好了以后,就可以在任何目录下新建一个virtual-environment(我更喜欢叫:盗梦空间),当然一般我习惯在项目的边上创建一个$project_name-env ...

more ...