Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

August 09 2011

Requests: HTTP for Humans — Requests v0.5.1 documentation

Most existing Python modules for sending HTTP requests are extremely verbose and cumbersome. Python’s builtin urllib2 module provides most of the HTTP capabilities you should need, but the api is thoroughly broken. It requires an enormous amount of work (even method overrides) to perform the simplest of tasks.

Things shouldn’t be this way. Not in Python.

>>> r = requests.get('https://api.github.com', auth=('user', 'pass'))
>>> r.status_code
200
>>> r.headers['content-type']
'application/json'

See the same code, without Requests.

Requests allow you to send HEAD, GET, POST, PUT, PATCH, and DELETE HTTP requests. You can add headers, form data, multipart files, and parameters with simple Python dictionaries, and access the response data in the same way. It’s powered by urllib2, but it does all the hard work and crazy hacks for you.

November 24 2010

High Scalability - High Scalability - Strategy: Biggest Performance Impact is to Reduce the Number of HTTP Requests

educe the size and frequency of network communications, which will make your pages load faster, which will improve performance enough that you can use HTTPS all the time, which will make you safe and secure on-line, which is a good thing.

July 22 2010

ImperialViolet - Overclocking SSL

In January this year (2010), Gmail switched to using HTTPS for everything by default. Previously it had been introduced as an option, but now all of our users use HTTPS to secure their email between their browsers and Google, all the time. In order to do this we had to deploy no additional machines and no special hardware. On our production frontend machines, SSL/TLS accounts for less than 1% of the CPU load, less than 10KB of memory per connection and less than 2% of network overhead. Many people believe that SSL takes a lot of CPU time and we hope the above numbers (public for the first time) will help to dispel that.

ImperialViolet - Overclocking SSL

In January this year (2010), Gmail switched to using HTTPS for everything by default. Previously it had been introduced as an option, but now all of our users use HTTPS to secure their email between their browsers and Google, all the time. In order to do this we had to deploy no additional machines and no special hardware. On our production frontend machines, SSL/TLS accounts for less than 1% of the CPU load, less than 10KB of memory per connection and less than 2% of network overhead. Many people believe that SSL takes a lot of CPU time and we hope the above numbers (public for the first time) will help to dispel that.
Reposted byn0g n0g

January 14 2010

http debugging with tshark [http] [tcpdump] [debugging] [sniffing] [wireshark] [tshark]

Very useful for http debugging(tshark is part of wireshark): # tshark -i eth0 -F visual -f "tcp port 80" -p -R http.request -N mntC In detail: -i interface -F display format -f tcpdump filter expression -p deactivates promiscuous mode -R wireshark display filter expression -N name resolve options
Tags: http debug cli

December 30 2009

Terrastore - Scalable, elastic, consistent document store.

Terrastore is a new-born document store which provides advanced scalability and elasticity features without sacrificing consistency.

Here are a few highlights:

  • Ubiquitous: based on the universally supported HTTP protocol.
  • Distributed: nodes can run and live everywhere on your network.
  • Elastic: you can add and remove nodes dynamically to/from your running cluster with no downtime and no changes at all to your configuration.
  • Scalable at the data layer: documents are partitioned and distributed among your nodes, with automatic and transparent re-balancing when nodes join and leave.
  • Scalable at the computational layer: query and update operations are distributed to the nodes which actually holds the queried/updated data, minimizing network traffic and spreading computational load.
  • Consistent: providing per-document consistency, you're guaranteed to always get the latest value of a single document, with read committed isolation for concurrent modifications.
  • Schemaless: providing a collection-based interface holding JSON documents with no pre-defined schema, you can just create your collections and put everything you want into.
  • Easy operations: install a fully working cluster in just a few commands and no XML to edit.
  • Features rich: support for push-down predicates, range queries and server-side update functions.

Read, participate, download and clone it!

October 05 2009

A database of open-source HTTP proxies written in python.

This site presents a list of open source HTTP proxies written in java and python, with comparison tables, so that you compare the proxies on a feature by feature basis. Further detail is available on each proxy: click it's name for more info.
Tags: proxy python http

June 18 2009

Slowloris HTTP DoS

Slowloris holds connections open by sending partial HTTP requests. It continues to send subsequent headers at regular intervals to keep the sockets from closing. In this way webservers can be quickly tied up. In particular, servers that have threading will tend to be vulnerable, by virtue of the fact that they attempt to limit the amount of threading they'll allow. Slowloris must wait for all the sockets to become available before it's successful at consuming them, so if it's a high traffic website, it may take a while for the site to free up it's sockets. So while you may be unable to see the website from your vantage point, others may still be able to see it until all sockets are freed by them and consumed by Slowloris. This is because other users of the system must finish their requests before the sockets become available for Slowloris to consume. If others re-initiate their connections in that brief time-period they'll still be able to see the site.
Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl