1.目录
linux下通过命令访问url的方式有多种,主要如下

2.1.elinks
elinks – lynx-like替代角色模式WWW的浏览器
例如: elinks –dump http://www.baidu.com

2.1.2.wget

这个会将访问的首页下载到本地

  1. [root@el5-mq2 ~]# wget http://www.baidu.com

3.3.curl
curl会显示出源码

  1. curl http://www.baidu.com/index.html

4.4.lynx

  1. lynx http://www.baidu.com

5.5.curl使用实践
现在有个需求,因为服务器在收集访问数据,抓取cookie中的value,模拟url访问时需要带上cookie参数,curl命令刚好能完成这个功能。
首先查看帮助:

  1. curl -h
  2. -b/–cookie <name=string/file> Cookie string or file to read cookies from (H)
  3. -c/–cookie-jar <file> Write cookies to this file after operation (H)
  4. create-dirs Create necessary local directory hierarchy
  5. crlf Convert LF to CRLF in upload
  6. crlfile <file> Get a CRL list in PEM format from the given file

可以使用-b参数来完成,具体使用如下:

  1. curl b key1=val1;key2=val2;”

或直接使用文件

  1. curl -b ./cookie.txt

编写测试实例:

  1. curl -b user_trace_token=20150518150621-02994ed9a0fb42d1906a56258e072fc4;LGUID=20150515135257-a33a769c-fac6-11e4-91ce-5254005c3644 http://10.10.26.164:1235/click?v=1&logtype=deliver&position=home_hot-0&orderid=10197777&userid=1942556&positionid=148&url=http%3a%2f%2fwww.lagou.com%2fjobs%2f317000.html%3fsource%3dhome_hot%26i%3dhome_hot-5&fromsite=http%3a%2f%2fwww.lagou.com%2fzhaopin%2fAndroid%3flabelWords%3dlabel%26utm_source%3dAD__baidu_pinzhuan%26utm_medium%3dsem%26utm_campaign%3dSEM&optime=2015-06-15_20:00:00

发现这样还是不可以,url附带的参数取不到。使用-d 参数传递url参数,使用-G 把请求方式配置为GET就OK了,如下:

  1. curl -b user_trace_token=20150518150621-02994ed9a0fb42d1906a56258e072fc4;LGUID=20150515135257-a33a769c-fac6-11e4-91ce-5254005c3644;LGSID=20150518150621-02994ed9a0fb42d1906a56258e072fc4;LGRID=20150617230732-4ea87972-1580-11e5-9a88-000c29653e90;” -d v=1&logtype=deliver&position=i_home-1&orderid=10197777&userid=1942556&positionid=148&url=http%3a%2f%2fwww.lagou.com%2fjobs%2f317000.html%3fsource%3dhome_hot%26i%3dhome_hot-5&fromsite=http%3a%2f%2fwww.lagou.com%2fzhaopin%2fAndroid%3flabelWords%3dlabel%26utm_source%3dAD__baidu_pinzhuan%26utm_medium%3dsem%26utm_campaign%3dSEM&optime=2015-06-15_20:00:00 -G http://10.10.26.164:1235/click

想要获得response返回的cookie怎么办,使用’-c’参数,同时可以使用-b filename用文件方式表示cookie,配合-c使用更方便
可以先用-c 命令生成一个cookie文件作为模板,再修改这个文件作为-b 参数的文件名。
使用如下:

  1. curl -b c1.txt -c c2.txt -d v=1&_v=j31&a=406405635&t=pageview&_s=1&dr=http%3a%2f%2fwww.sogou.com%2ftuguang&dl=http%3A%2F%2Fwww.lagou.com%2F%3futm_source%3dad_sougou_pingzhuan&ul=zh-cn&de=UTF-8&dt=%E6%8B%89%E5%8B%BE%E7%BD%91-%E6%9C%80%E4%B8%93%E4%B8%9A%E7%9A%84%E4%BA%92%E8%81%94%E7%BD%91%E6%8B%9B%E8%81%98%E5%B9%B3%E5%8F%B0&sd=24-bit&sr=1600×900&vp=1583×291&je=1&fl=18.0%20r0&_u=MACAAAQBK~&jid=&cid=1312768212.1431333683&tid=UA-41268416-1&z=1204746223 -G http://192.168.52.130:1234/collect

生成的c2.txt内容如下:

  1. # Netscape HTTP Cookie File
  2. # http://curl.haxx.se/docs/http-cookies.html
  3. # This file was generated by libcurl! Edit at your own risk.192.168.52.130 FALSE / FALSE 1757574737 user_trace_token 20150914151217-eedd019e-5aaf-11e5-8a69-000c29653e90
  4. 192.168.52.130 FALSE / FALSE 1442217595 LGSID 20150914152955-652a13c5-5ab2-11e5-846d-000c29653e90
  5. 192.168.52.130 FALSE / FALSE 1442217595 PRE_UTM
  6. 192.168.52.130 FALSE / FALSE 1442217595 PRE_HOST www.huxiu.com
  7. 192.168.52.130 FALSE / FALSE 1442217595 PRE_SITE http%3A%2F%2Fwww.huxiu.com%2Ftuguang
  8. 192.168.52.130 FALSE / FALSE 1442217595 PRE_LAND http%3A%2F%2Fwww.lagou.com%2F%3F
  9. 192.168.52.130 FALSE / FALSE 0 LGRID 20150914152955-652a1630-5ab2-11e5-846d-000c29653e90
  10. 192.168.52.130 FALSE / FALSE 1757574737 LGUID 20150914151217-eedd0624-5aaf-11e5-8a69-000c29653e90

直接请求:

  1. curl -X POST -d 'msg={"cmd":"ticker","userid":18252928,"platform":"mqtt"}' http://msg.live/msg-server/apiRequest

转载: http://www.yetdata.com/archives/128

分类: web

标签:   curl