Commit graph

171 commits

Author SHA1 Message Date
Tatsuhiro Tsujikawa
b18e62dba7 Treat 30X response without Location header field as error
This is required to make segmented download work.
2014-02-05 21:20:09 +09:00
Tatsuhiro Tsujikawa
ec4b729704 Better handling of 30X HTTP status codes
Reference: http://greenbytes.de/tech/tc/httpredirects/
2014-02-05 00:32:43 +09:00
Tatsuhiro Tsujikawa
e1e6bb1ec5 Don't fail multiple concurrent dl same file if auto-file-renaming is enabled 2013-11-08 00:59:57 +09:00
Tatsuhiro Tsujikawa
2aa8e01836 Fix typo 2013-11-06 23:40:59 +09:00
Nils Maier
e2700f50a5 Code cleanup in Http* 2013-09-20 23:06:49 +02:00
Nils Maier
8526ceeb45 Convert to autos and ranged loops 2013-08-21 05:56:17 +02:00
Tatsuhiro Tsujikawa
b6d84c0160 Use nullptr for std::shared_ptr and unique_ptr default parameter 2013-08-19 23:50:58 +09:00
Tatsuhiro Tsujikawa
6b397c8125 Use std::unique_ptr for CheckIntegrityEntry and FileAllocationEntry 2013-07-06 15:56:08 +09:00
Tatsuhiro Tsujikawa
57f1902ee1 Use std::unique_ptr for StreamFilter instead of std::shared_ptr 2013-07-04 00:39:11 +09:00
Tatsuhiro Tsujikawa
cb205a207c Use std::unique_ptr for HttpRequest instead of std::shared_ptr 2013-07-04 00:13:15 +09:00
Tatsuhiro Tsujikawa
9a38c102dc Use std::unique_ptr for HttpResponse instead of std::shared_ptr 2013-07-03 23:42:55 +09:00
Tatsuhiro Tsujikawa
47402c5f29 Refactor HttpConnection and its neighbor objects 2013-06-26 23:19:15 +09:00
Tatsuhiro Tsujikawa
fa9f3fb5a3 Wrap Command object in std::unique_ptr 2013-06-23 21:55:52 +09:00
Tatsuhiro Tsujikawa
07d270c87e Require -std=c++11 and use std::shared_ptr instead of SharedHandle 2013-06-22 01:10:38 +09:00
Tatsuhiro Tsujikawa
b9da9d4ed3 Use 64 bits random bytes as GID
This change replaces the current 64 bit sequential GID with 64 bits
random bytes GID in an attempt to support persistent GID. Internally,
the GID is stored as uint64_t. For human representation and RPC
interface, GID is represented as 16 bytes hex string. For console
readout, 16 bytes are too long, so it is abbreviated to first 6 bytes.
When querying GID in RPC calls, user can speicfy the prefix of GID as
long as the prefix is shared by more than 1 GID entries.
2012-12-16 17:29:01 +09:00
Tatsuhiro Tsujikawa
c904aa50fe Use uri_split instead of uri::parse where appropriate 2012-10-14 18:05:34 +09:00
Tatsuhiro Tsujikawa
8b0c701266 Removed trailing spaces 2012-10-01 23:52:22 +09:00
Tatsuhiro Tsujikawa
3258614033 Removed *Handle typedef 2012-09-28 23:27:46 +09:00
Tatsuhiro Tsujikawa
461a542c5e Rewritten DownloadHandlerConstants
DownloadHandlerConstants was simplified.  MIME type handling in Accept
header was also reworked.  DownloadContext's metalinkServerContacted_
is replaced with acceptMetalink_ and its boolean value is reverted.
RequestGroup and HttpRequest now do not hold vector of accepting
types.  HttpRequest has the flag acceptMetalink_ which will be set by
the same value of DownloadContext::accpetMetalink_ and if it is true,
Metalink MIME types are added to Accept header field.
2012-09-22 23:19:41 +09:00
Tatsuhiro Tsujikawa
1b874268a0 Use int64_t instead of off_t
Some classes such as DiskAdaptor, DiskWriter, FileAllocationIterator
and ChecksumValidator are left unchanged.
2012-06-25 23:35:24 +09:00
Tatsuhiro Tsujikawa
1c292f469e aria2 now doesn't assume download's completed just because file size matched
The only exception is zero-length file.  If server tells file is
zero-length and --checksum option is given, aria2 now correctly checks
its checksum. There is one known issue: If downloaded file is
zero-length file and .aria2 file exists, it will not be deleted on
successful verification, because .aria2 file is not loaded.
2012-01-08 17:46:03 +09:00
Tatsuhiro Tsujikawa
12988e5282 Replaced uint64_t with off_t or int64_t.
Since off_t is int64_t with LFS, we cannot take advantage of extra
capacity of uint64_t.
2011-12-09 21:39:43 +09:00
Tatsuhiro Tsujikawa
f860bf0d50 Use %lld to fomat a2_gid_t 2011-12-02 21:36:11 +09:00
Tatsuhiro Tsujikawa
02307cee0a Removed utf8ToNative from log message and exception message. 2011-08-08 21:46:10 +09:00
Tatsuhiro Tsujikawa
3879da592d In MinGW32, open file with UNICODE filename and print them in ANSI. 2011-08-04 21:43:02 +09:00
Tatsuhiro Tsujikawa
03f0774482 Consistent naming scheme for hash type and digest for DownloadContext. 2011-07-27 22:44:30 +09:00
Tatsuhiro Tsujikawa
7b86b294c9 Consistent naming scheme for hash type and digest. 2011-07-27 21:44:54 +09:00
Tatsuhiro Tsujikawa
a533437be6 Implemented Metalink/HTTP in HTTP download.
Link header fields from first Metalink server is utilized as described
in rfc6249. We only set digest from Digest header field to
DownloadContext only when PieceStorage is not initialized(in other
words, before file size is known). After PieceStorage is initialized,
Digest header field is used to check the value is the same in digest
in DownloadContext.  Current implementation only handles
rel=duplicate.
2011-07-27 20:54:21 +09:00
Tatsuhiro Tsujikawa
5a4a627d72 Remove Metalink content-type after first server response.
We don't accept Metalink content-type after first server response.
2011-07-25 00:03:13 +09:00
Tatsuhiro Tsujikawa
7230fbd2f7 Replaced HAVE_LIBZ with HAVE_ZLIB 2011-02-18 22:34:38 +09:00
Tatsuhiro Tsujikawa
1b9329c67c Accept HTTP 304 reply as success when If-None-Match request-header
field is specified using --header option.

When --conditional-get is used, --allow-overwrite is now required to
overwrite existing file.
2011-01-29 23:19:10 +09:00
Tatsuhiro Tsujikawa
63fe7874da Handle the situation when remote server returns Connection: close
after several pipelined requests.

We call Request::setMaxPipelinedRequest(1) if Connection: close is
received. Also call Request::supportsPersistentConnection(true) and
Request::setMaxPipelinedRequest(1) when closing the connection.
2011-01-16 18:12:14 +09:00
Tatsuhiro Tsujikawa
ea1b4b3ee5 Eliminated SocketCore::peekData() form HTTP/FTP downloads.
We introduced SocketRecvBuffer which buffers received bytes. Since
HTTP response header and response body are divided with \r\n, we have
to buffer up several bytes to find this delimiter. We use
SocketRecvBuffer to hold these bytes and only consumes header and
passes SocketRecvBuffer, which may contain head of response body, to
next Command.  Since FTPConnection doesn't use SocketCore::peekData(),
we left it as is.
2011-01-16 16:55:41 +09:00
Tatsuhiro Tsujikawa
0a2b1660fa Removed DownloadContext::dir_. Use PREF_DIR value instead. 2010-12-03 13:08:22 +09:00
Tatsuhiro Tsujikawa
ecbc05012c Added additional error_code::Values(9 through 14). 2010-11-28 17:29:32 +09:00
Tatsuhiro Tsujikawa
694fb307aa 2010-11-20 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Replaced StringFormat with fmt
2010-11-20 09:36:14 +00:00
Tatsuhiro Tsujikawa
580098eb49 2010-11-20 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Rewritten Logger interface. Logger now writes source file name and
	line number in log file.
2010-11-20 08:21:36 +00:00
Tatsuhiro Tsujikawa
d8d159ccd8 2010-11-15 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Replaced HttpHeader::responseStatus_ with HttpHeader::statusCode_.
	statusCode_ is of type int.
	* src/AbstractProxyResponseCommand.cc
	* src/FtpNegotiationCommand.cc
	* src/HttpHeader.cc
	* src/HttpHeader.h
	* src/HttpHeaderProcessor.cc
	* src/HttpResponse.cc
	* src/HttpResponse.h
	* src/HttpResponseCommand.cc
	* src/HttpSkipResponseCommand.cc
	* src/util.cc
	* src/util.h
	* test/HttpHeaderProcessorTest.cc
	* test/HttpHeaderTest.cc
	* test/HttpResponseTest.cc
	* test/UtilTest.cc
2010-11-15 12:52:03 +00:00
Tatsuhiro Tsujikawa
1372ac51ad 2010-11-14 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Added .cc file for classes/structs that only provided by header
	file. Defined non-POD classes' ctor, dtor in .cc file.  Moved
	implementation code in header file to .cc file for major
	classes/strucsts.
2010-11-14 07:17:55 +00:00
Tatsuhiro Tsujikawa
93e5dbed32 2010-11-12 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Removed SharedHandle::isNull(). Instead we added operator* and
	operator unspecified_bool_type. Removed use of WeakHandle and
	replaced with raw pointer.
2010-11-12 12:48:48 +00:00
Tatsuhiro Tsujikawa
bcf4593bda 2010-11-11 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Create SharedHandle outside of function call to avoid unexpected
	memory leak.
	* src/AnnounceList.cc
	* src/FtpNegotiationCommand.cc
	* src/HttpResponseCommand.cc
	* src/RequestGroup.cc
	* src/RequestGroupMan.cc
	* src/UTMetadataPostDownloadHandler.cc
	* src/download_helper.cc
2010-11-11 05:12:37 +00:00
Tatsuhiro Tsujikawa
236e64cb25 2010-10-30 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Use unnamed namespace instead of static keyword.
	* src/AbstractCommand.cc
	* src/AdaptiveURISelector.cc
	* src/Base64.cc
	* src/BitfieldMan.cc
	* src/BtDependency.cc
	* src/ConsoleStatCalc.cc
	* src/ContentTypeRequestGroupCriteria.cc
	* src/CookieStorage.cc
	* src/DHTMessageFactoryImpl.cc
	* src/DHTRoutingTableDeserializer.cc
	* src/DefaultBtAnnounce.cc
	* src/DefaultBtProgressInfoFile.cc
	* src/DefaultPeerStorage.cc
	* src/DefaultPieceStorage.cc
	* src/DownloadCommand.cc
	* src/DownloadEngine.cc
	* src/EpollEventPoll.cc
	* src/ExpatMetalinkProcessor.cc
	* src/ExpatXmlRpcRequestProcessor.cc
	* src/FileEntry.cc
	* src/HttpRequest.cc
	* src/HttpRequestCommand.cc
	* src/HttpResponseCommand.cc
	* src/KqueueEventPoll.cc
	* src/LongestSequencePieceSelector.cc
	* src/MetalinkParserStateV3Impl.cc
	* src/MetalinkParserStateV4Impl.cc
	* src/MultiDiskAdaptor.cc
	* src/MultiUrlRequestInfo.cc
	* src/OptionParser.cc
	* src/PeerSessionResource.cc
	* src/PortEventPoll.cc
	* src/Request.cc
	* src/RequestGroupMan.cc
	* src/SelectEventPoll.cc
	* src/SessionSerializer.cc
	* src/SimpleLogFormatter.cc
	* src/Sqlite3CookieParser.cc
	* src/TrackerWatcherCommand.cc
	* src/XML2SAXMetalinkProcessor.cc
	* src/Xml2XmlRpcRequestProcessor.cc
	* src/XmlRpcMethod.cc
	* src/XmlRpcMethodImpl.cc
	* src/XmlRpcResponse.cc
	* src/base32.cc
	* src/bencode2.cc
	* src/bittorrent_helper.cc
	* src/download_helper.cc
	* src/main.cc
	* src/messageDigest.cc
	* src/option_processing.cc
	* src/util.cc
	* test/AnnounceListTest.cc
	* test/BtRegistryTest.cc
	* test/DHTBucketTest.cc
	* test/DHTRoutingTableTest.cc
	* test/DefaultBtAnnounceTest.cc
	* test/FileEntryTest.cc
	* test/FtpConnectionTest.cc
	* test/MSEHandshakeTest.cc
	* test/MagnetTest.cc
	* test/XmlRpcMethodTest.cc
	* test/array_funTest.cc
2010-10-30 16:02:15 +00:00
Tatsuhiro Tsujikawa
6b74a27586 2010-10-19 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Fixed the bug that file allocation is enabled in HTTP even if
	--file-allocation=none is specified.
	* src/HttpResponseCommand.cc
2010-10-19 13:46:51 +00:00
Tatsuhiro Tsujikawa
d956ea0b70 2010-10-02 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Non-UTF8 filenames are now percent-encoded.  For example, filename
	for http://example.org/%90%A2%8AE will be %90%A2%8AE because it is
	Shift_JIS. The comments and name in .torrent file in XML-RPC
	response are percent-encoded if they are not UTF-8.
	* src/FtpNegotiationCommand.cc
	* src/HttpRequestCommand.cc
	* src/HttpResponseCommand.cc
	* src/XmlRpcMethodImpl.cc
	* src/bittorrent_helper.cc
	* src/util.cc
	* src/util.h
	* test/BittorrentHelperTest.cc
	* test/UtilTest.cc
2010-10-02 07:54:43 +00:00
Tatsuhiro Tsujikawa
a27968beda 2010-09-15 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Fixed the bug that a file gets overwritten if -V is given and no
	hash is provided. Fixed the bug that --dry-run leads download
	error. Added RequestGroup::createCheckIntegrityEntry() which
	correctly creates CheckIntegrityEntry objects and open files based
	on -V option and the existence of control file.
	* src/AbstractCommand.cc
	* src/AbstractCommand.h
	* src/ChecksumCheckIntegrityEntry.cc
	* src/DownloadContext.cc
	* src/DownloadContext.h
	* src/FtpNegotiationCommand.cc
	* src/HttpResponseCommand.cc
	* src/PieceHashCheckIntegrityEntry.cc
	* src/RequestGroup.cc
	* src/RequestGroup.h
	* src/RequestGroupEntry.cc
	* src/RequestGroupEntry.h
2010-09-15 11:46:25 +00:00
Tatsuhiro Tsujikawa
1906832f43 2010-09-13 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Fixed compile error without zlib
	* src/HttpResponseCommand.cc
2010-09-13 13:31:09 +00:00
Tatsuhiro Tsujikawa
7f9e70e5c8 2010-09-13 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Updated doc
	* src/FtpNegotiationCommand.cc
	* src/HttpResponseCommand.cc
	* src/RequestGroup.cc
2010-09-13 11:53:38 +00:00
Tatsuhiro Tsujikawa
97953dd1ae 2010-09-13 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Fixed the bug that error occurs when downloading zero-length file.
	* src/FtpNegotiationCommand.cc
	* src/HttpResponseCommand.cc
2010-09-13 11:47:04 +00:00
Tatsuhiro Tsujikawa
9eaf3ccb0f 2010-09-11 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Replaced V_TRUE with A2_V_TRUE. Replaced V_FALSE with A2_V_FALSE.
	* src/AbstractCommand.cc
	* src/HttpResponseCommand.cc
	* src/Option.cc
	* src/OptionHandlerFactory.cc
	* src/OptionHandlerImpl.h
	* src/TrackerWatcherCommand.cc
	* src/download_helper.cc
	* src/main.cc
	* src/prefs.cc
	* src/prefs.h
	* test/AuthConfigFactoryTest.cc
	* test/DefaultBtProgressInfoFileTest.cc
	* test/DownloadHelperTest.cc
	* test/HttpRequestTest.cc
	* test/OptionHandlerTest.cc
	* test/UTMetadataPostDownloadHandlerTest.cc
	* test/XmlRpcMethodTest.cc
2010-09-11 12:48:03 +00:00
Tatsuhiro Tsujikawa
80edde0205 2010-09-11 Tatsuhiro Tsujikawa <t-tujikawa@users.sourceforge.net>
Use uri::parse instead of Request::setUri() where just URI parser
	is needed.
	* src/AbstractCommand.cc
	* src/AdaptiveURISelector.cc
	* src/DefaultBtAnnounce.cc
	* src/FeedbackURISelector.cc
	* src/FileEntry.cc
	* src/HttpResponseCommand.cc
	* src/ProtocolDetector.cc
	* src/Request.cc
	* src/RequestGroupMan.cc
2010-09-11 12:09:19 +00:00