First script execution

[Yazur-admin]

Here is the error message I get with the command:

./check_haproxy.rb -u http://172.16.100.254/haproxy/haproxy_stats.php?haproxystats=1 -U *** -P ***

Traceback (most recent call last):
-- 2: from ./check_haproxy.rb:148:in `<\main>'

-- 1: from ./check_haproxy.rb:148:in `\each'

./check_haproxy.rb:158:in block in <\main>': undefined method zip' for nil:NilClass (NoMethodError)

[Yazur-admin]

Ruby version: ruby 2.5.5p157 (2019-03-15 revision 67260) [x86_64-linux-gnu]
Linux version: Debian GNU/Linux 10 (buster)

[benprew]

It looks like it can't find the csv header. Also, the script expects that it will have to append ;csv to the url to get the csv formatted output from haproxy.  The output from your url should look like this:

# pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,comp_in,comp_out,comp_byp,comp_rsp,lastsess,last_chk,last_agt,qtime,ctime,rtime,ttime,agent_status,agent_code,agent_duration,check_desc,agent_desc,check_rise,check_fall,check_health,agent_rise,agent_fall,agent_health,addr,cookie,mode,algo,conn_rate,conn_rate_max,conn_tot,intercepted,dcon,dses,
http-in,FRONTEND,,,2,100,100,16481008,3333252352,91243059717,8853370,0,987385,,,,,OPEN,,,,,,,,,1,2,0,,,,0,2,0,166,,,,0,6074933,400045,1362497,5635,8853045,,2,148,16586258,,,0,0,0,0,,,,,,,,,,,,,,,,,,,,,http,,2,166,16602559,2273510,0,0,
http-in,IPv4-direct,,,2,100,100,9264268,1438253974,5996570742,8790375,0,247,,,,,OPEN,,,,,,,,,1,2,1,,,,3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,http,,,,,,0,0,

Here's an example:

http://demo.1wt.eu/;csv

[Yazur-admin]

Thank you for your help.
I did have a bad URL.
I still have a problem but this time, in the certificate "/usr/lib/ruby/2.5.0/net/protocol.rb:44:in `connect_nonblock': SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain) (OpenSSL::SSL::SSLError)".

I tried to add the self-signed certificate in /usr/local/share/ca-certificates/ and run the command "update-ca-certificates".
Then adding the -k option doesn't change anything.

[Yazur-admin]

It looks like it can't find the csv header. Also, the script expects that it will have to append ;csv to the url to get the csv formatted output from haproxy.  The output from your url should look like this:

# pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,comp_in,comp_out,comp_byp,comp_rsp,lastsess,last_chk,last_agt,qtime,ctime,rtime,ttime,agent_status,agent_code,agent_duration,check_desc,agent_desc,check_rise,check_fall,check_health,agent_rise,agent_fall,agent_health,addr,cookie,mode,algo,conn_rate,conn_rate_max,conn_tot,intercepted,dcon,dses,
http-in,FRONTEND,,,2,100,100,16481008,3333252352,91243059717,8853370,0,987385,,,,,OPEN,,,,,,,,,1,2,0,,,,0,2,0,166,,,,0,6074933,400045,1362497,5635,8853045,,2,148,16586258,,,0,0,0,0,,,,,,,,,,,,,,,,,,,,,http,,2,166,16602559,2273510,0,0,
http-in,IPv4-direct,,,2,100,100,9264268,1438253974,5996570742,8790375,0,247,,,,,OPEN,,,,,,,,,1,2,1,,,,3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,http,,,,,,0,0,

Here's an example:

http://demo.1wt.eu/;csv

I disabled the https while I was doing the tests.
I always get the same error

My haproxy is a package installed on pfsense, could this be a problem?
Knowing that I arrive on the same page as you, whether I mention my admin and *** or without mentioning them.

[Yazur-admin]

[Yazur-admin]

To correct the error:
`connect_nonblock': SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain) (OpenSSL::SSL::SSLError).
I deleted the && RUBY_VERSION = 1.9
In the line if option.insecure_ssl && RUBY_VERSION = 1.9
open_opts[:ssl_verify_mode] = OpenSSL::SSL::VERIFY_NONE

[Yazur-admin]

[benprew]

Hi @Yazur-admin, can you try running this version of check_haproxy.rb with the --debug flag:

https://raw.githubusercontent.com/benprew/nagios-checks/personal/bprew/debug-output/check_haproxy.rb

It should print out the lines it is receiving from haproxy, that should give you some insight into what is causing the issue. I suspect that it's an authentication issue, that is working in your browser, but is not work when using check_haproxy.

Also, this version has the insecure_ssl fix you mention above.

Thanks

[Yazur-admin]

As you said in your first message the script doesn't find the CSV...
You think it can't authenticate and therefore reaches the pfsense home page that requires authentication and not the page containing the CSV?

Nobody uses this Haproxy script with Pfsense?

[Yazur-admin]

The -P and -U are not taken into account or does not work, the error message is the same.

[benprew]

The username and password that check_haproxy uses is used for basic auth, I suspect that pfsense doesn't use basic auth, do you know how authentication works in pfsense? Is it session/cookie based, or can you provide some kind of login token?

[Yazur-admin]

@benprew

Thank you for your response.
However, I don't really know what the basic method means to you.
I don't think Pfsense can allow connection with a connection token. There is always an association "Username" "Password" to be filled in.

We can use a RADIUS / LDAP server to connect but we always have to enter both "Username" and "Password" information.

[benprew]

Hi @Yazur-admin, thanks for replying. Sorry about the confusion, Basic Auth is a specific type of authentication, I should've clarified what I meant by that.

In this case it sounds like there is some of kind of session/cookie/etc that pfSense is storing after you login. I don't have pfSense set up anywhere, so I can't test it, but if you can provide the requests and responses that are needed, I could add them to check_haproxy

[Yazur-admin]

I would love to help you add this feature, but I have no idea how I can help you.

I've done some research on the internet regarding authentication on the pfsense webui page but I can't find any useful documentation.

Maybe I can find directly on the pfsense server the script that displays the pfsense authentication page and send you this script in .txt format.

[Yazur-admin]

index.php_PFSENSE.txt

Is this the right script?

Because I have a pretty big script list in reality:

[Yazur-admin]

After thinking about it, I don't think that the script sent is the login script but rather the home page once logged in.

[benprew]

Hey @Yazur-admin, I was hoping that would be enough info to replicate the auth process, but it doesn't look like it.

I'll probably need to test against a running version of pfsense. It looks like it runs on FreeBSD, is there a linux version I could run in docker or something similar?

If not, do you have a test instance I could test against?

Thanks

[benprew]

Or, if you have a simple command-line tool that logs into pfsense and I could look at the source code, I could probably replicate it here.

[Yazur-admin]

I took a look at the pfSense "Netgate" forum and one of the developers says
"No. It is its own complete operating system. It can be virtualized by a hypervisor, but not run in a container."

I try to set up a pfsense in the afternoon with public ip and SSH access.

I'll keep you informed!

[Yazur-admin]

@benprew I just sent you the accesses for the test server.
Thanks for your help, if you need to create a Linux VM to test the haproxy please let me know.

Thanks :) !

[benprew]

Thanks @Yazur-admin, I was able to script logging in, I'm trying to figure out a good way to incorporate it into the check_haproxy script

[Yazur-admin]

Thanks @Yazur-admin, I was able to script logging in, I'm trying to figure out a good way to incorporate it into the check_haproxy script

That's great, thank you.
I'm waiting for your script to test it in production.

[benprew]

Ok @Yazur-admin I created a script and put it in this repo: https://github.com/benprew/pfsense-login

You'll need to download it and the Gemfile and the run bundle install to install the dependencies.

Then, you can use it with check_haproxy in a bash script like this:

#!/bin/bash
COOKIE=$(cd pfsense-login && bundle exec login.rb -u "$URL" -U "$USERNAME" -P "$PASSWORD")
cd nagios-checks
./check_haproxy.rb --cookie "$COOKIE" -u "$HAPROXY_URL"

Where you fill in the values for $URL, $USERNAME, $PASSWORD and $HAPROXY_URL

[Yazur-admin]

Thank you very much !
It's great, we should be able to adapt the code to be able to use it in a monitoring server.

That is to say we should be able to write in clear text the authentication credentials in the code, because on pfsense we can create a user who can only see the haproxy stat page.

And use only check_haproxy commands without having to authenticate.

In other words, you would have to be able to put together the "pfsense authentication" code snippet with the "check_haproxy" one to be able to use this in a monitoring server like librenms.

Thank you very much for everything you've already done!
If you don't have time I'll understand.

[benprew]

Hi @Yazur-admin , I'm not sure I understand what you mean? Are you not able to use the 2 scripts together?

[Yazur-admin]

/usr/lib/ruby/2.5.0/net/protocol.rb:44:in `connect_nonblock': SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain) (OpenSSL::SSL::SSLError)

Problem with the certificate as it is self-signed.
You should either ignore the warning as in the other script.
Or add the certificate to the trusted certificates on the server.
I've already tried the second method but it doesn't work. I must be doing something wrong.

[Yazur-admin]

To return to my previous message:

With the simple command ./check_haproxy.rb --cookie "$COOKIE" -u "$HAPROXY_URL", I should be able to retrieve the haproxy information to supervise them in my LibreNMS server.

I can't execute other commands before, it would have to be automatic.
And so it would be great to have a line in the "check_haproxy.rb" code that executes a bash file with the commands needed for authentication.
That way, no more need to enter the credentials by hand.

[Yazur-admin]

I don't know how to write in Ruby, I have only bash and powershell notions.
But you should get the script from "login.rb", put it in check_haproxy.rb.
Replace the OptionParser -u -U and -P by -u = ""; -U = "" and -P="";
And I, for my part, put the right information between the quotation marks.

Or much simpler :

Add a line in the script "login.rb" which executes a bash file and in this bash file I would put this:

#!/bin/bash COOKIE=$(cd pfsense-login && bundle exec login.rb -u "$URL" -U "$USERNAME" -P "$PASSWORD") cd nagios-checks ./check_haproxy.rb --cookie "$COOKIE" -u "$HAPROXY_URL"

[Yazur-admin]

Here I have succeeded with the help of my fellow developer.
Thanks a lot :) !

You can see the changes made to the script with: https://text-compare.com/fr/

Here is the final code: https://code.empreintesduweb.com/14540.html

[benprew]

@Yazur-admin That's great! I'm glad you were able to get it working!