Logging full HTTP url requests
Clash Royale CLAN TAG#URR8PPP
up vote
0
down vote
favorite
I am looking to see how I could possibly log all http requests that are done by any client on my network, my network is my own personal home network and I am the only user of the network. The http requests should only be logged if the client has been setup to use the server on my network that is running a proxy of some description, otherwise it would just connect as normal and not via the proxy.
I don't need to do any caching or restricting access to specific hosts, only the logging of the hostnames and the requests for files.
I tried tinyproxy, but this was only logging the domain and not the full path, I tried squid also, but I couldn't find much help on how to configure this on what I was trying to achieve, it may be that my google fu isn't that good.
This is a partial log from tinyproxy.
CONNECT Feb 25 20:23:10 [31896]: Connect (file descriptor 6): 192.168.0.51 [192.168.0.51]
CONNECT Feb 25 20:23:10 [31896]: Request (file descriptor 6): CONNECT vortex-win.data.microsoft.com:443 HTTP/1.1
INFO Feb 25 20:23:10 [31896]: No upstream proxy for vortex-win.data.microsoft.com
CONNECT Feb 25 20:23:10 [31896]: Established connection to host "vortex-win.data.microsoft.com" using file descriptor 7.
INFO Feb 25 20:23:10 [31896]: Not sending client headers to remote machine
ERROR Feb 25 20:24:11 [31896]: readbuff: recv() error "Connection reset by peer" on file descriptor 6
INFO Feb 25 20:24:11 [31896]: Closed connection between local client (fd:6) and remote client (fd:7)
Am open to suggestions as what I could use to achieve the above, I am using debian stretch as the linux distribution.
linux debian proxy http
asked Feb 25 at 19:13
AeroMaxx
64
add a comment |Â
up vote
0
down vote
favorite
I am looking to see how I could possibly log all http requests that are done by any client on my network, my network is my own personal home network and I am the only user of the network. The http requests should only be logged if the client has been setup to use the server on my network that is running a proxy of some description, otherwise it would just connect as normal and not via the proxy.
I don't need to do any caching or restricting access to specific hosts, only the logging of the hostnames and the requests for files.
I tried tinyproxy, but this was only logging the domain and not the full path, I tried squid also, but I couldn't find much help on how to configure this on what I was trying to achieve, it may be that my google fu isn't that good.
This is a partial log from tinyproxy.
CONNECT Feb 25 20:23:10 [31896]: Connect (file descriptor 6): 192.168.0.51 [192.168.0.51]
CONNECT Feb 25 20:23:10 [31896]: Request (file descriptor 6): CONNECT vortex-win.data.microsoft.com:443 HTTP/1.1
INFO Feb 25 20:23:10 [31896]: No upstream proxy for vortex-win.data.microsoft.com
CONNECT Feb 25 20:23:10 [31896]: Established connection to host "vortex-win.data.microsoft.com" using file descriptor 7.
INFO Feb 25 20:23:10 [31896]: Not sending client headers to remote machine
ERROR Feb 25 20:24:11 [31896]: readbuff: recv() error "Connection reset by peer" on file descriptor 6
INFO Feb 25 20:24:11 [31896]: Closed connection between local client (fd:6) and remote client (fd:7)
Am open to suggestions as what I could use to achieve the above, I am using debian stretch as the linux distribution.
linux debian proxy http
asked Feb 25 at 19:13
AeroMaxx
64
You are not being specific into want you want, I was going to advisesquid. Give example of logs, and tell us why they are not useful for you.
– Rui F Ribeiro
Feb 25 at 19:23
1
How did you manage not to find squid-cache.org/Doc/config/access_log ?
– Gerard H. Pille
Feb 25 at 19:42
@GerardH.Pille I have since found that, but I find squid very difficult to use for a first time user, and was hoping for something that wasn't quite as heavy and just did the one thing I needed and not lots of other things that I would never need. Currently having major difficulty even connecting to the proxy despite setting up the correct acl I believe.
– AeroMaxx
Feb 25 at 21:31
Squid out of the box should do what you need. But I've only installed one ever, reverse proxies are more in demand. Upload your config somewhere, I'll have a look. Please mark your modifications.
– Gerard H. Pille
Feb 26 at 1:14
add a comment |Â
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am looking to see how I could possibly log all http requests that are done by any client on my network, my network is my own personal home network and I am the only user of the network. The http requests should only be logged if the client has been setup to use the server on my network that is running a proxy of some description, otherwise it would just connect as normal and not via the proxy.
I don't need to do any caching or restricting access to specific hosts, only the logging of the hostnames and the requests for files.
I tried tinyproxy, but this was only logging the domain and not the full path, I tried squid also, but I couldn't find much help on how to configure this on what I was trying to achieve, it may be that my google fu isn't that good.
This is a partial log from tinyproxy.
CONNECT Feb 25 20:23:10 [31896]: Connect (file descriptor 6): 192.168.0.51 [192.168.0.51]
CONNECT Feb 25 20:23:10 [31896]: Request (file descriptor 6): CONNECT vortex-win.data.microsoft.com:443 HTTP/1.1
INFO Feb 25 20:23:10 [31896]: No upstream proxy for vortex-win.data.microsoft.com
CONNECT Feb 25 20:23:10 [31896]: Established connection to host "vortex-win.data.microsoft.com" using file descriptor 7.
INFO Feb 25 20:23:10 [31896]: Not sending client headers to remote machine
ERROR Feb 25 20:24:11 [31896]: readbuff: recv() error "Connection reset by peer" on file descriptor 6
INFO Feb 25 20:24:11 [31896]: Closed connection between local client (fd:6) and remote client (fd:7)
Am open to suggestions as what I could use to achieve the above, I am using debian stretch as the linux distribution.
linux debian proxy http
asked Feb 25 at 19:13
AeroMaxx
64
I am looking to see how I could possibly log all http requests that are done by any client on my network, my network is my own personal home network and I am the only user of the network. The http requests should only be logged if the client has been setup to use the server on my network that is running a proxy of some description, otherwise it would just connect as normal and not via the proxy.
I don't need to do any caching or restricting access to specific hosts, only the logging of the hostnames and the requests for files.
I tried tinyproxy, but this was only logging the domain and not the full path, I tried squid also, but I couldn't find much help on how to configure this on what I was trying to achieve, it may be that my google fu isn't that good.
This is a partial log from tinyproxy.
CONNECT Feb 25 20:23:10 [31896]: Connect (file descriptor 6): 192.168.0.51 [192.168.0.51]
CONNECT Feb 25 20:23:10 [31896]: Request (file descriptor 6): CONNECT vortex-win.data.microsoft.com:443 HTTP/1.1
INFO Feb 25 20:23:10 [31896]: No upstream proxy for vortex-win.data.microsoft.com
CONNECT Feb 25 20:23:10 [31896]: Established connection to host "vortex-win.data.microsoft.com" using file descriptor 7.
INFO Feb 25 20:23:10 [31896]: Not sending client headers to remote machine
ERROR Feb 25 20:24:11 [31896]: readbuff: recv() error "Connection reset by peer" on file descriptor 6
INFO Feb 25 20:24:11 [31896]: Closed connection between local client (fd:6) and remote client (fd:7)
Am open to suggestions as what I could use to achieve the above, I am using debian stretch as the linux distribution.
linux debian proxy http
asked Feb 25 at 19:13
AeroMaxx
64
edited Feb 25 at 20:27
asked Feb 25 at 19:13
AeroMaxx
64
asked Feb 25 at 19:13
AeroMaxx
64
asked Feb 25 at 19:13
AeroMaxx
64
64
You are not being specific into want you want, I was going to advisesquid. Give example of logs, and tell us why they are not useful for you.
– Rui F Ribeiro
Feb 25 at 19:23
1
How did you manage not to find squid-cache.org/Doc/config/access_log ?
– Gerard H. Pille
Feb 25 at 19:42
@GerardH.Pille I have since found that, but I find squid very difficult to use for a first time user, and was hoping for something that wasn't quite as heavy and just did the one thing I needed and not lots of other things that I would never need. Currently having major difficulty even connecting to the proxy despite setting up the correct acl I believe.
– AeroMaxx
Feb 25 at 21:31
Squid out of the box should do what you need. But I've only installed one ever, reverse proxies are more in demand. Upload your config somewhere, I'll have a look. Please mark your modifications.
– Gerard H. Pille
Feb 26 at 1:14
add a comment |Â
You are not being specific into want you want, I was going to advisesquid. Give example of logs, and tell us why they are not useful for you.
– Rui F Ribeiro
Feb 25 at 19:23
1
How did you manage not to find squid-cache.org/Doc/config/access_log ?
– Gerard H. Pille
Feb 25 at 19:42
@GerardH.Pille I have since found that, but I find squid very difficult to use for a first time user, and was hoping for something that wasn't quite as heavy and just did the one thing I needed and not lots of other things that I would never need. Currently having major difficulty even connecting to the proxy despite setting up the correct acl I believe.
– AeroMaxx
Feb 25 at 21:31
Squid out of the box should do what you need. But I've only installed one ever, reverse proxies are more in demand. Upload your config somewhere, I'll have a look. Please mark your modifications.
– Gerard H. Pille
Feb 26 at 1:14
You are not being specific into want you want, I was going to advise
squid. Give example of logs, and tell us why they are not useful for you.– Rui F Ribeiro
Feb 25 at 19:23
You are not being specific into want you want, I was going to advise
squid. Give example of logs, and tell us why they are not useful for you.– Rui F Ribeiro
Feb 25 at 19:23
1
1
How did you manage not to find squid-cache.org/Doc/config/access_log ?
– Gerard H. Pille
Feb 25 at 19:42
How did you manage not to find squid-cache.org/Doc/config/access_log ?
– Gerard H. Pille
Feb 25 at 19:42
@GerardH.Pille I have since found that, but I find squid very difficult to use for a first time user, and was hoping for something that wasn't quite as heavy and just did the one thing I needed and not lots of other things that I would never need. Currently having major difficulty even connecting to the proxy despite setting up the correct acl I believe.
– AeroMaxx
Feb 25 at 21:31
@GerardH.Pille I have since found that, but I find squid very difficult to use for a first time user, and was hoping for something that wasn't quite as heavy and just did the one thing I needed and not lots of other things that I would never need. Currently having major difficulty even connecting to the proxy despite setting up the correct acl I believe.
– AeroMaxx
Feb 25 at 21:31
Squid out of the box should do what you need. But I've only installed one ever, reverse proxies are more in demand. Upload your config somewhere, I'll have a look. Please mark your modifications.
– Gerard H. Pille
Feb 26 at 1:14
Squid out of the box should do what you need. But I've only installed one ever, reverse proxies are more in demand. Upload your config somewhere, I'll have a look. Please mark your modifications.
– Gerard H. Pille
Feb 26 at 1:14
add a comment |Â
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f426520%2flogging-full-http-url-requests%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
You are not being specific into want you want, I was going to advise
squid. Give example of logs, and tell us why they are not useful for you.– Rui F Ribeiro
Feb 25 at 19:23
1
How did you manage not to find squid-cache.org/Doc/config/access_log ?
– Gerard H. Pille
Feb 25 at 19:42
@GerardH.Pille I have since found that, but I find squid very difficult to use for a first time user, and was hoping for something that wasn't quite as heavy and just did the one thing I needed and not lots of other things that I would never need. Currently having major difficulty even connecting to the proxy despite setting up the correct acl I believe.
– AeroMaxx
Feb 25 at 21:31
Squid out of the box should do what you need. But I've only installed one ever, reverse proxies are more in demand. Upload your config somewhere, I'll have a look. Please mark your modifications.
– Gerard H. Pille
Feb 26 at 1:14