Perlbal::Manual::Management(3) Managing Perlbal on-the-fly

VERSION

Perlbal 1.78.

DESCRIPTION

How to configure a Perlbal "management" service for on-the-fly configuration and debugging.

READ ME FIRST

Please read Perlbal::Manual::Configuration first for a better explanation on how to configure Perlbal. This document will make much more sense after reading that.

Setting up a management service

You can create a management service in the following way: 

    CREATE SERVICE mgmt
        SET role   = management
        SET listen = 127.0.0.1:16000
    ENABLE mgmt

Consulting information with a browser

If you access the management service (in this case, 127.0.0.1 on port 16000) with a browser you'll reach a page with information on the services that are enabled. You're also able to click those services and reach further information on each of them.

Managing Perlbal via telnet

If you telnet to the management service you get a connection that allows you manage your Perlbal's instance. 

    $ telnet 127.0.0.1 16000
    Trying 127.0.0.1...
    Connected to 127.0.0.1.
    Escape character is '^]'.

You now have access to several commands (which are case insensitive):

Admin Commands

create pool <name>
create service <name>
Creates a pool or a service. 

    create pool my_new_pool
    create service my_new_service
disable <service>
Disables a service (stops listening). 

    disable my_service

Note that you can not disable "management" services.

enable <service>
Enables a service (starts listening). 

    enable my_service
header <service> insert <header>:<value>
header <service> remove <header>
Inserts or removes headers from the request before they're passed on to the backend.

When adding headers you must state the value: 

    header my_service insert x-myamazingheader:myamazingvalue

When removing headers you don't need the state any value, the header will be removed regardless of the value it holds: 

    header my_service remove Connection

Note: the usage of "x-" in the beginning of your header is not required, but the convention is that non-standard headers be prefixed with "x-".

load <plugin>
Loads a Perlbal plugin. 

    load AccessControl
plugins
Lists plugins
pool <name> add <ip[:port]>
pool <name> remove <ip[:port]>
Allows you to add or remove nodes from a service. 

    pool add my_service 127.0.0.1:1337
    pool remove my_service 127.0.0.1:1337

Note that adding a node that already exists or removing a non-existing node do not result in an error message.

Also note that the port number defaults to :80 if omitted.

reproxy_state
Dumps the state of reproxy status. 

    reproxy_state
    SERVER max_reproxy_connections = 0
server <param> = <value>
Sets a server parameter, where param is one of:
  • max_reproxy_connections = <value>

    Maximum number of concurrent connections to the backends.

  • max_reproxy_connections("ip") = <value>

    Maximum number of concurrent connections to one specific backend.

  • max_connections = <value>

    Sets the value of "RLIMIT_NOFILE" (maximum number of open files for this process).

    Requires BSD::Resource. Also, Perlbal must be run under a superuser.

  • nice_level = <value>

    Sets the "nice" level for the process.

  • aio_mode = <value>

    One of "none", "linux" for Linux::AIO, or "ioaio" for IO::AIO.

    This controls how disk IO is done asynchronously. Highly recommended to use Linux::AIO or IO::AIO for webserving or reproxying files. For purely reverse proxy or only reproxying URLs, none is fine.

  • aio_threads = <value>

    Number of child threads doing disk IO. Use between 2 and 50.

  • track_obj = <value>

    Developer option to track objects.

  • pidfile = <value>

    Filename to write pidfile to (no pidfile if not specified).

  • crash_backtrace = <value>

    1 or 0 indicating whether to perform a backtrace while the server is crashing.

set [<pool_name>] <param> = <value>
Sets a property on a pool. If the pool was just created, specifying the pool name is optional.

Setting the file to autoload nodes from (Perlbal will periodically check the file for updates): 

    SET my_pool nodefile = /path/to/file

Unsetting the file to autoload nodes from (note that this does not remove current members): 

    SET my_pool nodefile = none

"undef", "null", "" and '' are interpreted just like "none".

Note that manually modifying the pool (via POOL ADD or POOL REMOVE) will disable the periodic checking of the nodefile.

Setting the load balancing method: 

    SET pool balance_method = 'random'

Yes, we do realize that "random" is the only method currently available, but hey.

set [<service_name>] <param> = <value>
Sets a property on a service. If the service was just created, specifying the service name is optional.

See Perlbal::Manual::LoadBalancer, Perlbal::Manual::ReverseProxy and Perlbal::Manual::WebServer for the list of available parameters.

show service [<service_name>]
show pool [<pool_name>]
List all services: 

    show service

Show details of a service: 

    show service my_service

Lists all pools, nodes and services using them: 

    show pool

Show the members of a pool: 

    show pool my_pool
shutdown [graceful]
Shuts down the server (you'll also lose your telnet session).

Can be used instantly, killing all active connections: 

    shutdown

Using the parameter "graceful" listening sockets are closed and perlbal stays alive until clients bleed off. 

    shutdown graceful
unload <plugin>
Unloads a plugin. 

    unload AccessControl
use <service>
use <pool>
Sets the implied service or pool for future operations.

For instance, after: 

    use my_pool

You can just type: 

    pool add 127.0.0.1:6000

Note that creating a service or pool also sets it as the implied service.

xs
xs [enable <module>]
xs [disable <module>]
Show status of XS modules loaded: 

    xs

Turn on an already-loaded XS module: 

    xs enable module_name

Turn off an already-loaded XS module: 

    xs disable module_name

Diagnostic Commands

aio
Shows Perlbal::AIO stats.
backends
Lists the open connections to the backends. 

    backends
    127.0.0.1:3080 1
    127.0.0.1:3081 2

In the above example, there are 3 open connections (they may be being used or they may simply be open according to the value of "connect_ahead").

dumpconfig
Shows the current configuration for each service.

If a service uses a plugin that supports "dumpconfig", that plugin's configuration is also dumped (see "dumpconfig" under Perlbal::Manual::Plugins for more information).

fd
Shows the maximum number of file descriptors and how many are currently in use. 

    fd
    max 1024
    cur 8
gladiator
Requires Devel::Gladiator.

Iterate's Perl's internal memory structures and can be used to enumerate all the currently live SVs.

This can be used to hunt leaks and to profile memory usage.

help
Displays the list of available commands.
leaks
leaks <code>
Iterates over active objects.

If an argument is specified, it is treated as code with $_ being the reference to the object.

Shows objects that might have been leaked.

mime
Lists known mime types. 

    mime
    css text/css
    doc application/msword
    ...
node <ip[:port]>
Dumps information on a specific node. 

    node 127.0.0.1:8181
    127.0.0.1:8181 attempts 10
    127.0.0.1:8181 lastattempt 1290461126

Calling "node" with a parameter is the same as calling "nodes".

nodes
Dumps information on all nodes. 

    nodes
    127.0.0.1:8181 attempts 10
    127.0.0.1:8181 lastattempt 1290461126
    127.0.0.1:8081 connects 19
    127.0.0.1:8081 lastconnect 1290461127
    127.0.0.1:8081 attempts 19
    127.0.0.1:8081 responsecodes 200 11
    127.0.0.1:8081 lastattempt 1290461127
noverify
Shows the amount of time left for each node marked as noverify.

If the "verify_backend" parameter is set to a true value, perlbal tries to send an "OPTIONS" command to a node before sending it the actual client request. If the node doesn't support the "OPTIONS" command, the node is added to an internal hash so that an "OPTIONS" command is not issued for new requests during the next 60 seconds. 

    noverify
    127.0.0.1:8081 42

In the above example, node 127.0.0.1:8081 is on that list and still has 42 seconds until an "OPTIONS" command is tried again.

If the time is a negative value it means that the "OPTIONS" command will be issued before the next request.

obj
Shows objects count in scope.

The environment variable "DEBUG_OBJ" must be set to a true value (see "Environment Variables" under Perlbal::Manual::Configuration for more information).

pending
Shows pending backend connections by service, node, and age.
proc
Shows CPU usage, current time, pid, and total requests processed.
prof on
prof off
prof data
Enables/disables profiling or dumps profile data.
queues
Shows status of all queues (normal, high priority and low priority) for all enabled services. 

    queues
    service_mywebsite-normal.age 1
    service_mywebsite-normal.count 8
    service_mywebsite-highpri.age 0
    service_mywebsite-highpri.count 0
    service_mywebsite-lowpri.age 0
    service_mywebsite-lowpri.count 0
reload <plugin_name>
Reloads a plugin that supports reloading.
socks [all]
socks summary
Lists all sockets or shows a summary of the sockets (by default lists all sockets). 

    socks
       fd    age
        3   748s Perlbal::ClientManage(R): open to 85.245.86.253:52248
        4  1003s Perlbal::TCPListener(R): open: listening on 82.102.30.112:80 for service 'ws'
        6  1003s Perlbal::TCPListener(R): open: listening on 0.0.0.0:60000 for service 'mgmt'
    socks summary
        1 Perlbal::ClientManage
        2 Perlbal::TCPListener
    Aggregate write buffer: 0.0k
                Open files: 0
state changes
Lists recent state changes of requests.

For instance, while a request is being processed this command may issue something like: 

    state changes
    Perlbal::ClientProxy=HASH(0x12d7ec28): reading_headers, wait_backend, backend_req_sent, wait_res
    Perlbal::BackendHTTP=HASH(0x12d75f30): connecting, bored, sending_req, wait_res
    wait_res 2

And as soon as the request is processed: 

    state changes
    Perlbal::BackendHTTP=HASH(0x12d75f30): connecting, bored, sending_req, wait_res, xfer_res, closed
    closed 1

Note: This functionality depends on the environment variable "PERLBAL_TRACK_STATES" being set to a true value.

states
states <service>
Shows how many sockets of which type and in which state currently exist for all services. 

    states
    Perlbal::ClientProxy draining_res 1

If a service is specified, shows the information for that service only. 

    states ws
    Perlbal::BackendHTTP wait_res 1
    Perlbal::ClientProxy wait_res 1
track
Dumps objects tracked, sorted by age. 

    track
    15s Perlbal::HTTPHeaders=HASH(0x1a043a50): Perlbal::HTTPHeaders::clone, Perlbal::BackendHTTP::assign_client, Perlbal::Service::register_boredom, Perlbal::BackendHTTP::event_write, Danga::Socket::EpollEventLoop, (eval), Perlbal::run
    15s Perlbal::HTTPHeaders=HASH(0x19594fe0): Perlbal::HTTPHeaders::new, Perlbal::Socket::read_headers, Perlbal::Socket::read_response_headers, Perlbal::BackendHTTP::event_read_waiting_options, Perlbal::BackendHTTP::event_read, Danga::Socket::EpollEventLoop, (eval), Perlbal::run
    15s Perlbal::HTTPHeaders=HASH(0x1a043d80): Perlbal::HTTPHeaders::new, Perlbal::Socket::read_headers, Perlbal::Socket::read_response_headers, Perlbal::BackendHTTP::event_read, Danga::Socket::EpollEventLoop, (eval), Perlbal::run
    15s Perlbal::BackendHTTP=HASH(0x1a044608): Perlbal::BackendHTTP::new, Perlbal::Service::spawn_backends, Perlbal::Service::note_bad_backend_connect, Perlbal::BackendHTTP::event_err, Danga::Socket::EpollEventLoop, (eval), Perlbal::run
    15s Perlbal::BackendHTTP=HASH(0x1a042588): Perlbal::BackendHTTP::new, Perlbal::Service::spawn_backends, Perlbal::Service::note_bad_backend_connect, Perlbal::BackendHTTP::verify_failure, Perlbal::BackendHTTP::event_read_waiting_options, Perlbal::BackendHTTP::event_read, Danga::Socket::EpollEventLoop, (eval), Perlbal::run
    15s Perlbal::BackendHTTP=HASH(0x1a044488): Perlbal::BackendHTTP::new, Perlbal::Service::spawn_backends, Perlbal::Service::request_backend_connection, Perlbal::ClientProxy::request_backend, Perlbal::ClientProxy::handle_request, Perlbal::ClientProxy::event_read, Danga::Socket::EpollEventLoop, (eval), Perlbal::run

For this to work you must have the environment variable "DEBUG_OBJ" set to a true value and the server option "track_obj" on: 

    server track_obj = 1
    OK
uptime
version
Shows the time the server was started, the current uptime and Perlbal's version.
varsize
Debug management command to track size of internal data structures. 

    varsize
    svc-ws-bored_backends [] 2
    svc-ws-pending_connects {} 2
      {127.0.0.1:3080}  =
      {127.0.0.1:3081}  = Perlbal::BackendHTTP=HASH(0x1cfd2490)
verbose on
verbose off
Turns verbose mode on or off. 

    verbose on
    load AccessControl
    OK
    verbose off
    load AccessControl
    verbose on
    OK
    load AccessControl
    OK

Yes, verbose mode only activates the printing of an OK message upon completion of some commands such as "load". Errors are displayed regardless of verbose status.

CAVEAT

Note that this functionality doesn't implement any sort of authentication. If you can telnet to the address you specify, you can access the information and change things on the fly.

Some alternatives to authentication are commonly employed:

  • specifying the address as being 127.0.0.1 means that no one outside the machine can telnet to the management service;
  • setting the management address as a private IP available only through a VPN, for instance, will allow you to manage these accesses on the VPN instead.

LAST SEARCHED