Streaming HTTP uploads with PHP ?

The problem: http classes provided by PHP packages such as PEAR and Zend framework will, when uploading a file via HTTP, read the complete file contents into memory before uploading the file. This hogs memory and can lead to errors (not enough memory available) in case of large files being transferred.

PHP's curl crust seemed to be the solution, but only when you're using the PUT method. When using the POST method another issue has me stumped.

Possible solution: PEAR has a new Http_Request2 class, which seems to support streaming HTTP uploads through its socket adapter. Currently in alpha.

Comments

Post a Comment

Popular posts from this blog

Handling control characters (escaping) in python for json and mysql

JSON loads() and dumps(): how control characters are handled The json lib escapes control characters when you dump(s) a python object into a json string. import json aa = "String with\ttab and\nnewline" print "1. Py string with control chars, unescaped: '%s'" % aa aa = json.dumps(aa, indent = 2, encoding="utf8") # dumping py-obj into json-string will cause control chars to be escaped print "\n2. JSON Encoded String with control chars, escaped: '%s'" % aa aa = json.loads(aa) # loading json-string into py-obj will leave control chars unescaped print "\n3. Py object (= string) with control chars, unescaped: '%s'" % aa Output: 1. Py string with control chars, unescaped: 'String with tab and newline' 2. JSON Encoded String with control chars, escaped: '"String with\ttab and\nnewline"' 3. Py object (= string) with control chars, unescaped: 'String with tab and newline' Note that json.l...
Read more

python port sniffer with pcapy and impacket

The code below will sniff all network traffic (use a filter to limit the sniffer to only certain packets) during 1000 milliseconds and print some info about it. It requires 2 python modules: pcapy for capturing the packets impacket for analyzing the packet content Note that this sample uses pcapy's dispatch() method and relies on its ability to return automatically after a timeout expires. This functionality is not available on all platforms (e.g. not on linux). import os, logging, socket import pcapy, impacket, impacket.ImpactDecoder ETHERNET_MAX_FRAME_SIZE = 1518 PROMISC_MODE = 0 class Packet(object): def __init__(self, bytes, sniffed_bytes, timestamp, proto_id, src_ip, tgt_ip, src_port, tgt_port, msgs): """ @brief Initialize data. """ super(Packet, self).__init__() self.bytes = bytes self.sniffed_bytes = sniffed_bytes self.src_ip = src_ip self.tgt_ip = tgt_ip self.src_port = src_port self.tgt_port = tgt_port ...
Read more

Django field, form and model validation process

Django's form and field validation process can be best understood by looking at the code in django's BaseForm class (in django.forms.forms) - see below. See also this blog post for a graphic overview. The form.full_clean() method is called by django at the start of the validation process (by form.is_valid(), usually immediately after the view receives the posted data). full_clean() performs a 3 phase validation , with each phase corresponding to a call of one of the protected methods below: self._clean_fields() : Field validation This method iterates over the form fields: For each field, it calls the Widget's value_from_datadict() method, passing the complete post data in a QueryDict. The Widget's task is to retrieve its own value from this QueryDict and return it. The returned value is then passed to the Field's own clean() method. This method does the initial validation (e.g. checking if the value is not empty), without touching the db. If the Field's...
Read more