Page tree
Skip to end of metadata
Go to start of metadata
Content of this page

NetCurl is and is not...

  • Licensed under the Apache License, Version 2.0
  • NetCurl was not an attempt to rebuild the wheel.
  • NetCurl was back in time a regular (yet another) curl-wrapper, with the ability to test simpler input- and output data that was returned from proxies (socks, TOR, etc). The primary goal of the module was to be able to both auto scan different kind of websites for open proxies and then test them. As soon as there was proxies detected, they was added to DNS blacklists..
  • NetCurl is a preconfigured wrapper of standard libraries, created to get started as quick as possible without any further knowledge than the phrase "I'd like to get the content of XXX site, in the format of YYY, regardless of what they run and return". Instead of setting up classes, calls and methods, NetCurl activates a state of high verbosity, so that the developer can pick the data freely.
  • NetCurl can, if WordPress is present, switch over to the WP_HTTP-class, instead of using internal functions for the calls.
  • NetCurl can, if GuzzleHttp is present, switch over to GuzzleHttp to utilize and automatically set up a communication link via this external library. If curl is available, Guzzle will use curl as the first option (which makes this a bit useless). However, if curl is not present, NetCurl can instead utilize the streams driver provided in Guzzle. NetCurl will become a GuzzleWrapper, so to speak.
  • NetCurl follows the PHP and curl development and handles certificates with a best practice. It can however, override this and lower security by completely ignore SSL validations.
  • NetCurl parses and splits up data in sections of header, response code, body and a pre-parsed data container. The header can normally be viewed as an array, the body contains the received data content as is (raw, untouched), while the parsed data tries to automatically detect content and convert it to readable arrays/objects.
  • NetCurl supports SoapClient. It's done by the class Tornevall_simpleSoap. This module has dependencies to xml and SoapClient. SoapCalls are handled automatically from the base class: Adding ?wsdl to the URL fetched by NetCurl, will automatically initiate a session for soap. The call can also be sent without ?wsdl, but in that case you have to tell the module to go SOAP.

  • NetCurl supports native basic authentication (curl) and enables similar support when going through a GuzzleStream. For curl, authentication comes for free, while you're pretty much on your own when going stream. NetCurl also makes sure that the SoapClient adopts authentication data set by the setAuthentication()-method.
  • NetCurl was back in time more focused to be a "network attack protection tool", rather than a functional network communications tool. Many things has happened since then.

 

Parsed responses

If you make a web request to a site that responds in json formatted code, the parsed content will be returned as an object when running getParsedContent(). If the request returns XML, the parsed content will make sure it is converted in the same way. By running NetCurl in an older state (setChain=off) the raw data containers can be accesses without the method getParsedContent().

NetCurl Parsing

  • By PHP serialized content (serialize/unserialize)
  • JSON
  • XML (Starting with SimpleXMLElements, the PEAR/PECL modules such as XmlSerializer/XMLUnserializer are also supported)
  • DOMDocument (Converts html documents into DOMDocument arrays, by three methods: nodes, closest tags and element ids)

NetCurl extra features

The name NetCurl was born from the fact that there are more than one module in this class file. A network module also resides in the single file, which is the primary reason why it is called NetCurl. NetCurl takes care of different network based features, like resolving hosts, making sure that ip-addresses is really ip-addresses, extracts proxy addresses from server variables, encodes base64-strings into URL-compatible strings, reads CIDR-formatted netmasks properly, etc, etc, etc. This part of the documentation has documentation of all those modules.

Installation

The easiest way to install netcurl is by running something like this:

1. Get composer.
2. Run composer:

composer require tornevall/tornelib-php-netcurl

Composer also adds the crypto module during the installation. API-docs can be found here, as this site is not yet updated with this.

Dependencies

NetCurl itself is built in a way to be as independent as possible. The list below contains some components that is good to have, to be as compatible with the world as possible.

  • Composer-based installations of netcurl will also install the Crypto module.
  • Curl is not required - however, if curl is not installed, it's important that you have at least one compatible module available (WP_HTTP, GuzzleHttp, or so)
  • SSL (OpenSSL) - Not having SSL available means that you won't be able to speak https
  • To make use of the SOAP components in NetCurl, XML libraries and SoapClient needs to be there. SoapClient uses Streams to fetch wsdl.

In apt-based systems, extra libraries can be installed with commands such as:

apt-get install php-curl php-xml

Internal drivers

DriverClassProvidesInstallationNotes

CURL

Tornevall_cURL

curlCurl must be installed 
SimpleSoap

Tornevall_SimpleSoap

SoapClient
(Streams)
  

Importable drivers

NetCurl primarily supports curl, as the name reveals. However, if you run wordpress or Guzzle, there is a small driver switch in NetCurl that makes it possible to survive without curl. Below, the supported drivers are listed and what they may provide for you.

DriverClassProvidesInstallationNotes
Wordpress
WP_HTTP
curl and streamsWP must be presentOn SOAP, SoapClient makes fallback to internal class
GuzzleHttp
GuzzleHttp\Client
GuzzleHttp\Handler\StreamHandler 
curl and streams

Add to composer.json:

"guzzlehttp/guzzle": "6.3.0"

On SOAP, SoapClient makes fallback to internal class

Driver overriders

Overriding the default drivers can be done by running for example:

$CURL = new Tornevall_cURL();
$CURL->setDriver(TORNELIB_CURL_DRIVERS::DRIVER_GUZZLEHTTP);
// or
$CURL->setDriver(TORNELIB_CURL_DRIVERS::DRIVER_GUZZLEHTTP_STREAM);

The best available driver could also be autodetected via the method setDriverAuto().

NetCurl Library Content

NetCurl contains a bunch of classes in a single file that in the beginning was supposed to make the library independent. The namespace is TorneLIB.

ClassDescriptionAPI Document
TorneLIB_NetworkHandler of any networking related actionsLink
MODULE_CURLThe magic class that does the work for you in all http-related actionsLink
Tornevall_SimpleSoapSoapClient-handlerLink
TORNELIB_NETCURL_EXCEPTIONSExceptions related to the libraryLink
CURL_AUTH_TYPESAvailable authentication types for use with password protected sitesLink
CURL_METHODSList of methods available in this library (Like GET, POST, PUT, DELETE)Link
CURL_POST_ASPrepared formatting for POST-content in this library (Also available from for example PUT)
Examples: json, xml, etc
Link
CURL_RESOLVERResolver methods that is available when trying to connect (ipv4/ipv6 based selectors).
This defines whether the resolver should prioritize ipv4 or ipv6 first.
Link
TORNELIB_CURL_DRIVERSAvailable http driver handlers - Internal (default), WordPress, Guzzle, etcLink
TORNELIB_CURL_ENVIRONMENTUsed for testingLink
TORNELIB_CURL_RESPONSETYPEResponsetypes (how they are returned - as arrays or objects)
Normally not used
Link
TORNELIB_CURLOBJECTIf you need to get the responses out as objects, this is the predefined objectLink
TorneLIB_Network_IP_ProtocolsAddress Types class (ipv4/ipv6)
This defined wheter the connector should prioritize ipv4 or ipv6 first
Link
TorneLIB_NetBitsBitmasking handler (will probably be moved out of this library soon)Link

 

NetCurl responses (Usage lite)

The list below shows the response content that are returned from a GET/POST/etc. You necessarily do not need to fetch the keys yourself; you can also use NetCurl internal methods to get each data key from the response. As of v6.0.18, NetCurl is chained by default. This means that chained (PHP 5.4+) commands can be used (like $LIB->doGet()->getParsedResponse()). Chaining can be disabled with $LIB->setChain(false); - however, chaining is automatically disabled if PHP 5.3 or lower are detected. The table below shows how the raw data content looks "unchained".

Response keyGet methodarray content value
codegetResponseCode()

The current HTTP Status code returned from the request (See https://en.wikipedia.org/wiki/List_of_HTTP_status_codes)

This response status code can be used to detect, for example 404 not found errors, or similar, of a page. By means, you may throw optional exceptions by checking this code state as you need.

header 

The full response header returned from the request as ...

Header array
array(
    'info' => array(),
    'full' => 'Full header as string'
);

... where info is an array with keys and values. Example:

InfoArray
array(
    "Server" => "Apache",
    "Strict-Transport-Security" => "max-age=63072000; includeSubDomains",
    "X-Frame-Options" => "SAMEORIGIN",
    "X-Powered-By" => "PHP/5.6.27"
);
bodygetResponseBody()The full returned body of the request
parsedgetParsedResponse()

If the body are recognized as specially formatted (json, xml, etc), the parsed array will transform into an object or an array, depending on the content.

Currently, the parser supports simpler modes like XML (SimpleXMLElement) and by this also RSS-feeds (unconfirmed), JSON and serialized data. The curl library also supports usage of the PEAR-package XML_Serializer.

  • No labels