Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Panel
titleContent of this page

Table of Contents

Class usage, manuals and howtos
Children Display
alltrue

NetCurl is and is not...

  • Licensed under the Apache License, Version 2.0
  • NetCurl was not an attempt to rebuild the wheel.
  • NetCurl was back in time a regular (yet another) curl-wrapper, with the ability to test simpler input- and output data that was returned from proxies (socks, TOR, etc). The primary goal of the module was to be able to both auto scan different kind of websites for open proxies and then test them. As soon as there was proxies detected, they was added to DNS blacklists..
  • NetCurl is a preconfigured wrapper of standard libraries, created to get started as quick as possible without any further knowledge than the phrase "I'd like to get the content of XXX site, in the format of YYY, regardless of what they run and return". Instead of setting up classes, calls and methods, NetCurl activates a state of high verbosity, so that the developer can pick the data freely.
  • NetCurl can, if WordPress is present, switch over to the WP_HTTP-class, instead of using internal functions for the calls.
  • NetCurl can, if GuzzleHttp is present, switch over to GuzzleHttp to utilize and automatically set up a communication link via this external library. If curl is available, Guzzle will use curl as the first option (which makes this a bit useless). However, if curl is not present, NetCurl can instead utilize the streams driver provided in Guzzle. NetCurl will become a GuzzleWrapper, so to speak.
  • NetCurl follows the PHP and curl development and handles certificates with a best practice. It can however, override this and lower security by completely ignore SSL validations.
  • NetCurl parses and splits up data in sections of header, response code, body and a pre-parsed data container. The header can normally be viewed as an array, the body contains the received data content as is (raw, untouched), while the parsed data tries to automatically detect content and convert it to readable arrays/objects.
  • NetCurl supports SoapClient. It's done by the class Tornevall_simpleSoap. This module has dependencies to xml and SoapClient. SoapCalls are handled automatically from the base class: Adding ?wsdl to the URL fetched by NetCurl, will automatically initiate a session for soap. The call can also be sent without ?wsdl, but in that case you have to tell the module to go SOAP.

  • NetCurl supports native basic authentication (curl) and enables similar support when going through a GuzzleStream. For curl, authentication comes for free, while you're pretty much on your own when going stream. NetCurl also makes sure that the SoapClient adopts authentication data set by the setAuthentication()-method.
  • NetCurl was back in time more focused to be a "network attack protection tool", rather than a functional network communications tool. Many things has happened since then.

 

...

titleParsed responses

...

Other documents

Exceptions handling

Compatibility span (Supported PHP versions)

This library should be compatible with at least PHP 5.3 up to PHP 7.2. However, running this module in older versions of PHP makes it more unreliable as PHP is continuosly developed. There are no guarantees that the module is fully functional from obsolete releases (like PHP 5.3 and most of the PHP 5.x-series is).

Requirements and dependencies

Some kind of a supported driver is needed. NetCURL was built for curl so it could be a great idea to have curl available in your system. The goal with the module is however to pick up the best available driver in your system.

Supported drivers

  • curl
  • soap (SoapClient with XML)
  • Guzzle
  • Wordpress 

Future plans for independence:

  • Streams
  • Sockets

Dependencies and installation

As netcurl is built to be independently running dependencies is not necessesary required. To reach full functionality the list below might be good to have available.

  • Installation: Composer. NetCURL can be bundled/manually downloaded, but the best practice is to install via composer. Otherwise you're a bit on your own.
  • SSL (OpenSSL): Not having SSL available means that you won't be able to speak https
  • SOAP: To make use of the SOAP components in NetCurl, XML libraries and SoapClient needs to be there. SoapClient uses Streams to fetch wsdl.

XML, CURL, SOAP

In apt-based systems, extra libraries can be installed with commands such as:


No Format
apt-get install php-curl php-xml php-soap


The module installation itself

This is the recommended way (and only officially supported) of installing the package.

1. Get composer.
2. Run composer:

No Format
composer require tornevall/tornelib-php-netcurl

Composer also adds the crypto module during the installation. API-docs can be found here, as this site is not yet updated with this.

NetCurl Parsing

  • By PHP serialized content (serialize/unserialize)
  • JSON
  • XML (Starting with SimpleXMLElements, the PEAR/PECL modules such as XmlSerializer/XMLUnserializer are also supported)
  • DOMDocument (Converts html documents into DOMDocument arrays, by three methods: nodes, closest tags and element ids)

...

The name NetCurl was born from the fact that there are more than one module in this class file. A network module also resides in the single file, which is the primary reason why it is called NetCurl. NetCurl takes care of different network based features, like resolving hosts, making sure that ip-addresses is really ip-addresses, extracts proxy addresses from server variables, encodes base64-strings into URL-compatible strings, reads CIDR-formatted netmasks properly, etc, etc, etc. This part of the documentation has documentation of all those modules.

Installation

The easiest way to install netcurl is by running something like this:

1. Get composer.
2. Run composer:

No Format
composer require tornevall/tornelib-php-netcurl

Composer also adds the crypto module during the installation. API-docs can be found here, as this site is not yet updated with this.

Dependencies

NetCurl itself is built in a way to be as independent as possible. The list below contains some components that is good to have, to be as compatible with the world as possible.

  • Composer-based installations of netcurl will also install the Crypto module.
  • Curl is not required - however, if curl is not installed, it's important that you have at least one compatible module available (WP_HTTP, GuzzleHttp, or so)
  • SSL (OpenSSL) - Not having SSL available means that you won't be able to speak https
  • To make use of the SOAP components in NetCurl, XML libraries and SoapClient needs to be there. SoapClient uses Streams to fetch wsdl.

In apt-based systems, extra libraries can be installed with commands such as:

No Format
apt-get install php-curl php-xml

Internal drivers

DriverClassProvidesInstallationNotes

CURL

TornevallMODULE_cURLCURL

curlCurl must be installed 
SimpleSoap

TornevallMODULE_SimpleSoapSOAP

SoapClient
(Streams)
  

Importable drivers

...

Overriding the default drivers can be done by running for example.

Say that you'd prefer to use Guzzle over CURL. First of all, this is kind of double building the network module as guzzle primarily uses CURL to run. So doing this will make you build a wrapper over a wrapper, instead of going directly through curl.

First of all, you need to get the guzzle package. This is done by adding this row into your composer.json-file (or use the composer require statement).

Code Block
"guzzlehttp/guzzle": "^6.3"

When this is done, use composer install or composer update and start adding your code:

Code Block
$CURL = new TornevallMODULE_cURLCURL();
$CURL->setDriver(TORNELIBNETCURL_CURLNETWORK_DRIVERS::DRIVER_GUZZLEHTTP);
// or
$CURL->setDriver(TORNELIBNETCURL_CURLNETWORK_DRIVERS::DRIVER_GUZZLEHTTP_STREAM);

The best available driver could also be autodetected via the method setDriverAuto().

...

Library Content

NetCurl contains a bunch of classes in a single file that in the beginning was supposed to make the library independent. The namespace is TorneLIB.

ClassDescriptionAPI Document
TorneLIBMODULE_NetworkNETWORKHandler of any networking related actionsLink
MODULE_CURLThe magic class that does the work for you in all http-related actionsLink
TornevallMODULE_SimpleSoapSOAPSoapClient-handlerLink
TORNELIB_

NETCURL_EXCEPTIONS

Exceptions related to the libraryLink
CURLNETCURL_AUTH_TYPESAvailable authentication types for use with password protected sitesLink
CURLNETCURL_POST_METHODSList of methods available in this library (Like GET, POST, PUT, DELETE)Link
CURLNETCURL_POST_ASDATATYPESPrepared formatting for POST-content in this library (Also available from for example PUT)
Examples: json, xml, etc
Link
CURLNETCURL_RESOLVERResolver methods that is available when trying to connect (ipv4/ipv6 based selectors).
This defines whether the resolver should prioritize ipv4 or ipv6 first.
Link
TORNELIB

NETCURL_

CURL

NETWORK_DRIVERS

Available http driver handlers - Internal (default), WordPress, Guzzle, etcLink
TORNELIBNETCURL_CURL_ENVIRONMENTUsed for testing (deprecated)Link
TORNELIBNETCURL_CURL_RESPONSETYPEResponsetypes (how they are returned - as arrays or objects)
Normally not used
Link
TORNELIB

NETCURL_

CURLOBJECT

HTTP_OBJECT

If you need to get the responses out as objects, this is the predefined objectLink
TorneLIB

NETCURL_

Network_

IP_

Protocols

PROTOCOLS

Address Types class (ipv4/ipv6)
This defined wheter the connector should prioritize ipv4 or ipv6 first
Link
TorneLIB

MODULE_

NetBits

NETBITS

Bitmasking handler (will probably be moved out of this library soon)Link

 

NetCurl responses (

...

Unchained)

The list below shows the response content that are returned from a GET/POST/etc. You necessarily do not need to fetch the keys yourself; you can also use NetCurl internal methods to get each data key from the response. As of v6.0.18, NetCurl is chained by default. This means that chained (PHP 5.4+) commands can be used (like $LIB->doGet()->getParsedResponse>getParsed()). Chaining can be disabled with $LIB->setChain(false); - however, chaining is automatically disabled if PHP 5.3 or lower are detected. The table below shows how the raw data content looks "unchained".

Response keyGet methodarray content value
codegetResponseCodegetCode()

The current HTTP Status code returned from the request (See https://en.wikipedia.org/wiki/List_of_HTTP_status_codes)

This response status code can be used to detect, for example 404 not found errors, or similar, of a page. By means, you may throw optional exceptions by checking this code state as you need.

header 

The full response header returned from the request as ...

Code Block
languagephp
titleHeader array
collapsetrue
array(
    'info' => array(),
    'full' => 'Full header as string'
);

... where info is an array with keys and values. Example:

Code Block
languagephp
titleInfoArray
collapsetrue
array(
    "Server" => "Apache",
    "Strict-Transport-Security" => "max-age=63072000; includeSubDomains",
    "X-Frame-Options" => "SAMEORIGIN",
    "X-Powered-By" => "PHP/5.6.27"
);


bodygetResponseBodygetBody()The full returned body of the request
parsedgetParsedResponsegetParsed()

If the body are recognized as specially formatted (json, xml, etc), the parsed array will transform into an object or an array, depending on the content.

Currently, the parser supports simpler modes like XML (SimpleXMLElement) and by this also RSS-feeds (unconfirmed), JSON and serialized data. The curl library also supports usage of the PEAR-package XML_Serializer.

...