Satellite Internet Forum

[imepals]

Guys i need your help on this.

I have notice that on my satellite link that my daily bandwidth utilisation hits 240kbps on a link which is a 256kbps symmetric SCPC link with a DVB modem that receives and a comtech 570L modem for transmit. My service provider keeps saying that its from my network but my router activity does not support that argument.

From my network setup, a router separates the vsat network from the local area network. this internal router (LAN) gets traffic from the LAN and delivers to the firewall which then passing traffic to the router hosting the vsat. On the internal router the network traffic is almost at zero level after office hours (that is when users have closed for the days work).

On the other hand, i noticed that during peak periods (when users are online) the software used to monitor (cacti) indicates that the link performance hits an average of 100kbps whereas at off peak it hits 200kbps and above, that is almost working on full capacity.

I would please need suggestions on this issues I have stated and what I can do at my end to trouble shoot my VSAT link so that i can present more facts to my service provider should the problem be from them.

Secondly I would like to know why my link is slow when uses are online. We have less than 70 PCs on this link in a fully and welled segmented network.  All users are on VLANs (about 5 VLANs) and we use high end switches for inter VLAN routing and 3550 CISCO switches at the distribution level, there are no hubs in this network.

Any more information on this network to help in resolving this problem will be made available. Thanks for your anticipated suggestions.

[Derek Bell]

Just a start. I hope others will add more replies.

If you have the IP module in the 570L there are statistics available about traffic both on the LAN side and the satellite side. Header and payload compression stats may also available. It would be interesting to see how the satellite uplink traffic corresponds to what is going into the modem. Then step back and investigate the traffic from the first router connected. If it is a Cisco you may be able to bridge the port and monitor all the traffic on a 'parallel' listen port. Otherwise just put a hub in the way to break out an ethernet. Run it into ethereal for a while and see where the packets are coming from and going to, and why.

The high uplink traffic during off peak period is strange. It might be a virus on a PC sending out emails at top speed all of the time, some server PC distributing MP3 files, a disk back up system dumping to some off-site back up device etc.  Does the firewall itself dump its disk to an offsite server every night? None of the above may apply, so please will others add their comments below ...

[Oasis Networks]

just couple of fast notes:

1. From your description, it sounds like your link does reach its limits: If you see on the MRTG (or any other sampling application you are using) that it reaches 240kbps, remember that it is just a sampling and the actual usage is always bit more. How much more - it depends on couple of factors, one of them is the sampling interval.

2. I agree with Derek that you should take a deep look for trojans and worms on all the PC's of your network. I suggest you to troubleshoot it doing this: First, connect all your machines to the network and turn them on, open all the applications (like explorer, messenger and so on) but don't do anything - don't browse or touch the PC. Check the bandwidth usage. If you locate a problem, try to disconnect half of the machines to localize which machine(s) are contaminated. Once you will pinpoint the problem you will be able to solve it as well.

Good luck,
Nimrod

[imepals]

Thanks for your contributions so far.

For viruses, we have a Mcafee enterprise solution that is centrally managed for spam and viruses. Also there is no off-site backup or dumping of disk. All backups are manually done and during weekends (afternoon). All the switches and routers we have are Cisco products. The FW is Cisco PIX. The 570L does not have an IP module.

Is it possible for ethereal to monitor traffic of a router interface from my PC? if yes, how? I have installed it but cannot find a place where I can specify the IP or MAC address of an interface I want to monitor except that of the NIC on the PC.

Also what are the major things I should check for to ascertain if my VSAT link is in proper shape?

Nimrod I have worked on your suggestion in locating a contaminated PC but I am yet to conclude on it because I have over 70 PCs distributed in 5 floors. This will take some time to complete.

[Oasis Networks]

Well you can check the physical parameters, like Ebno on both sides, CRCs and so on.

Another two checks you can do is:

1. Try to ping -w 500 and -l of, lets say, 1000 for a long time and see how much time outs do you have and what is the pattern of the time outs.
2. Check the counters on your router, erase them and check if they are increasing.

[imepals]

This is that result I get from

Ping 66.94.234.13 -l 1000 -t

Ping statistics for 66.94.234.13:
Packets: Sent = 62, Received = 0, Lost = 62 (100% loss),

and for ping 66.94.234.13 -w 500 -t

Ping statistics for 66.94.234.13:
Packets: Sent = 62, Received = 0, Lost = 62 (100% loss)

I have also reset the counters on the router interfaces. My EbNo is 16db, how do i check the CRC?

[Derek Bell]

Regarding ethereal, you need to connect your monitor PC in parallel with the traffic you want to monitor. Two ways:

1. Find the existing cable to be monitored and make a break out. Put a passive hub with a side cable to your PC.
2. If one end of the cable goes to or comes from a Cisco router or switch and you know how to configure it, it may be possible to use a spare port and bridge it across in parallel with the port to be monitored. Cisco have some good traffic monitoring facilities, but be very careful about any Cisco reconfiguration. If you have a switch on each floor then I guess you might be able to use a spare port to do a monitor on any particular PC connected. If you have one main switch connected to each floor then at that switch you may be able to use a spare port to monitor any whole floor. The connection between your firewall and the satellite modem may need solution 1 above, using a passive hub.

Regarding the Eb/No of your transmit carrier (from the 570L) you need to call the hub and see what is the reading at their end at different times of day. It is possible that your transmit carrier is low power (moisture in cable, antenna moved slightly etc). It could get worse at night if the satellite moves out of the beam during that period. This can happen if the dish is mispointed. During clear sky conditions at both ends you should have a good rain margin of perhaps 4 or 5 dB. During clear sky conditions at both ends try reducing your transmit level by 3 dB. The network should continue operating normally. If is fails then you have less than 3 dB margin which is not good enough. Contact the hub and re-peak your dish and adjust your transmit level, while talking to them, till the Eb/No is correct at the hub.

[Oasis Networks]

This is that result I get from

Ping 66.94.234.13 -l 1000 -t

Ping statistics for 66.94.234.13:
Packets: Sent = 62, Received = 0, Lost = 62 (100% loss),

and for ping 66.94.234.13 -w 500 -t

Ping statistics for 66.94.234.13:
Packets: Sent = 62, Received = 0, Lost = 62 (100% loss)

I have also reset the counters on the router interfaces. My EbNo is 16db, how do i check the CRC?

For the Ebno it is OK. You will need to check the Ebno at your service provider though. For the ping, I can ping it very well, I guess the ping is blocked by your firewall.

[imepals]

The firewall is not blocking the ping. If i run a normal ping you get this

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\>ping 66.94.234.13 -t

Pinging 66.94.234.13 with 32 bytes of data:

Request timed out.
Reply from 66.94.234.13: bytes=32 time=4191ms TTL=47
Request timed out.
Request timed out.
Request timed out.
Reply from 66.94.234.13: bytes=32 time=3217ms TTL=47
Reply from 66.94.234.13: bytes=32 time=2812ms TTL=47
Reply from 66.94.234.13: bytes=32 time=3052ms TTL=46
Reply from 66.94.234.13: bytes=32 time=3337ms TTL=46
Reply from 66.94.234.13: bytes=32 time=3491ms TTL=46
Request timed out.
Reply from 66.94.234.13: bytes=32 time=3355ms TTL=46
Reply from 66.94.234.13: bytes=32 time=3361ms TTL=46
Reply from 66.94.234.13: bytes=32 time=3568ms TTL=46
Reply from 66.94.234.13: bytes=32 time=2907ms TTL=46
Reply from 66.94.234.13: bytes=32 time=3331ms TTL=47
Reply from 66.94.234.13: bytes=32 time=2733ms TTL=47
Reply from 66.94.234.13: bytes=32 time=3356ms TTL=46
Reply from 66.94.234.13: bytes=32 time=3541ms TTL=46
Reply from 66.94.234.13: bytes=32 time=3654ms TTL=47
Reply from 66.94.234.13: bytes=32 time=3992ms TTL=47
Reply from 66.94.234.13: bytes=32 time=3694ms TTL=47
Reply from 66.94.234.13: bytes=32 time=3793ms TTL=47
Reply from 66.94.234.13: bytes=32 time=3506ms TTL=47
Reply from 66.94.234.13: bytes=32 time=3749ms TTL=47
Reply from 66.94.234.13: bytes=32 time=4334ms TTL=47
Request timed out.
Reply from 66.94.234.13: bytes=32 time=3808ms TTL=46

Ping statistics for 66.94.234.13:
Packets: Sent = 28, Received = 22, Lost = 6 (21% loss),
Approximate round trip times in milli-seconds:
    Minimum = 2733ms, Maximum = 4334ms, Average = 3490ms
Control-C
^C

[Derek Bell]

Using about 20 packets, I get a wide variety of ping times the other way to an IP address ending in .178

Test 1. 646mS 3958mS average: 1090mS. No lost packets
Test 2. 1821mS 3498mS average 2656mS No lost packets
Test 2. 834mS 2598mS average 1693mS No lost packets

Try tracert also.

[pgannon]

It sounds as though you have a DVB/SCPC service, not SCPC/SCPC.  Your DVB download may be oversubscribed more than you expect.

I did not see any mention of TCP/HTTP Acceleration.  If you don't have TCP Acceleration it's quite normal that you would not be able to fully utilize the circuit.  The TCP protocol is a guaranteed delivery protocol.  It sends a little data, stops, waits for an ACK from the remote receiver, then doubles up the amount of data, sends, stops, etc.  In this manner it learns the speed and congestion of the link.  Since satellite delay is at least 1/2 second, the TCP protocol interprets this as a very slow or congested circuit.  Regardless of how much bandwidth you have, you are unlikely to get much more than about 70 - 90 Kbps per TCP session.  Of course multiple sessions will allow you to fill up the circuit, but given all the TCP overhead, it's unlikely you'll ever see full use of your circuit. With TCP Acceleration, you may get throughput in excess of the link speed because a lot of the TCP overhead is stripped off.

Comtech has an IP module that provides QoS and other features, but it does not include integrated TCP Acceleration.  For this you need an external device.  It also helps if your network operator is providing TCP/HTTP Acceleration on the DVB portion of the link.

HTTP Acceleration is also useful to speed up browsing.  Every single piece of content on a web page must go  through a 3-way handshake process.  Again, with the 1/2 second latency, this can really slow down web downloads between pieces of content.

See someone like Packeteer/Mentat for a TCP/HTTP Acceleration solution, or perhaps UDCast.  There are others out there, that I'm sure people on the forum can point you to.  Most 2-way services such as iDirect have this built in.

Hope this helps,
Pat  

[Maxim Usatov]

Gentlemen,

I am going to put it in a bit different perspective. 256 kbit/s Rx is way too small for 70 PCs in an office environment, even if it is all CIR. I would recommend a maximum of 5-10 simultaenously online PCs per a link providing 256 kbit/s throughput, depending on the usage patterns and environment. How many simultaneously online PCs do you have in average?

As for the pings, what was the network state when you did the ping test? Have you disconnected all the PCs or you ran the pings with the network connected? If second, then the link is overloaded. It will be slow if the usage won't be controlled.

Additionally, some providers have their DVB carriers are a bit oversubscribed to attract the customers with cheap pricing. Patrick is correct. Some providers offer 256 kbit/s CIR, however that's not a true CIR. That's probably why you can see it's operating at 200 kbit/s. If "256 kbit/s CIR" is not stated in SLA, the sharing factor could be even more.

What you may want to do in this case:

1) Squeeze the most out of your link. Implement HTTP acceleration and compression. This can be done with 3rd party accelerators as Pat mentioned. BusinessCom has it's own PEP (Performance Enhancing Proxy) which is an accelerator/compressor. We can deploy a trial version of PEP for you free of charge so you could see it in action. It doesn't require any proprietary hardware, you'll need a Linux server which will work as a transparent proxy for all the HTTP traffic.

2) Control over how the services are used in your network. You may want to implement smart traffic engineering solutions such as Packeteer or once again, BusinessCom TES-1000 unit to control traffic flows, deprioritize unwanted traffic, implement a throttler. That helps when the network is busy and budget is tight.

3) Add more bandwidth into the VSAT backbone. 256 kbit/s for 70 PCs is really too small. You may also want to consider changing platform to provide you with a broader BIR quota if your traffic is burstable. Probably for the same cash you're buying a 256/256 CIR link you could get something like 1024/256 BIR with 128/32 CIR which may perform better. This needs a deeper analysis though and it's hard to tell right away.

Please let me know if you would like me to assist you with the PEP trial. That may save you some kbit/s.

[imepals]

Thanks for all your response to this post. Maxim Usatov please do send me the PEP trial. I have also considered your suggestions but would like to know how to calculate the capacity if satellite you need for a given number of PCs. Over here like I stated before we have about 70 PCs, 50 of which can be online simultaneously.

Another problem I have just noticed is the high latency on my network. A round-trip delay returns at >3000ms sometimes 4000ms.

An investigation revealed that a ping from my fast-ethernet port to the serial port of my boarder router returns at >1000ms! These 2 interfaces are on the same router (cisco 3745 router).

Also in the past week i have observed from my Cacti software (a bandwidth utilization monitoring tool) that I am transmitting at over 307kpbs (I have not upgraded to this service rate) whereas i am subscribed for a bandwidth of 256/256kpbs. I have reported this to the service provider but nothing has been done about it and besides the latency is still high.

On the DVB, is there a way I can determine the overbooking or oversubscription rate from the DVB box or is it only the provider that has access to such information? And whats the main difference between DVB/SCPC and SCPC/SCPC?

Pinging yahoo.com [216.109.112.135] with 32 bytes of data

Reply from 216.109.112.135: bytes=32 time=3989ms TTL=53
Reply from 216.109.112.135: bytes=32 time=3960ms TTL=53
Reply from 216.109.112.135: bytes=32 time=4049ms TTL=53
Reply from 216.109.112.135: bytes=32 time=4288ms TTL=53
Request timed out.
Reply from 216.109.112.135: bytes=32 time=4332ms TTL=54
Request timed out.
Reply from 216.109.112.135: bytes=32 time=4274ms TTL=53
Request timed out.
Reply from 216.109.112.135: bytes=32 time=4094ms TTL=54
Reply from 216.109.112.135: bytes=32 time=3859ms TTL=54
Reply from 216.109.112.135: bytes=32 time=3658ms TTL=54
Reply from 216.109.112.135: bytes=32 time=3837ms TTL=53
Reply from 216.109.112.135: bytes=32 time=3771ms TTL=53
Reply from 216.109.112.135: bytes=32 time=4057ms TTL=53
Reply from 216.109.112.135: bytes=32 time=4157ms TTL=53
Reply from 216.109.112.135: bytes=32 time=3816ms TTL=54
Reply from 216.109.112.135: bytes=32 time=4015ms TTL=54
Request timed out.
Reply from 216.109.112.135: bytes=32 time=4127ms TTL=53

Ping statistics for 216.109.112.135:
    Packets: Sent = 20, Received = 16, Lost = 4 (20% loss)
Approximate round trip times in milli-seconds:
    Minimum = 3658ms, Maximum = 4332ms, Average = 4017ms

Expecting your professional advise as usual.

[Leopold Maurer]

70 clients on a 256k?  My goodness, talk about trying to do more with less.  That sounds painful.

I would venture to say that your link is beyond oversubscription.

[imepals]

The general idea here seems to portary that my network is overloaded. Can someone please suggest how I can calculate the bandwidth that will be appropriate for 70 online PCs including servers (file, print, mail and application servers) and also 2 VoIP phones?

Thank you in advance for your expert advise.