Unexpected maintenance 11/5/2012

Last month we moved our forum to a new server when the DDoS incident occured. We specifically ordered the server to be setup with a raid array. RAID (redundant array of independent disks) is a storage technology that combines multiple disk drive components into a logical unit.* This technology protects our files in case one of the drive fails preventing loss of data and gives us more time to change the failing drive. Unfortunately they did not setup the drives in raid and we only learn about this when the drive we are using failed tonight. After the drive failed our database immediately stopped working. We are now trying to recover the files onto a new drive. The retrieval process could take as long as 24hours. I know that is a very long time but we have no other option. Let's just hope that no important data is destroyed.

Thanks for your patience. :)

In the  meantime you can visit our facebook page if you want to chat with other PINOYDEN members.

How to change the default SSH port(22)

In this How-To we're going to walk you though changing the default SSH port on a Linux system.

The Secure Shell (SSH) Protocol by default uses port 22. Accepting this value does not make your system insecure, nor will changing the port provide a significant variance in security. However, changing the default SSH port will stop many automated attacks and a bit harder to guess which port SSH is accessible from. In other words, a little security though obscurity.

Steps to follow

Step 1: As root, use your favorite text editor (vi) to edit the sshd configuration file.

vi /etc/ssh/sshd_config

Step 2: Edit the line which states 'Port 22'. But before doing so, you'll want to read the note below. Choose an appropriate port, also making sure it not currently used on the system.

# What ports, IPs and protocols we listen for
Port 50683

Note: The Internet Assigned Numbers Authority (IANA) is responsible for the global coordination of the DNS Root, IP addressing, and other Internet protocol resources. It is good practice to follow their port assignment guidelines. Having said that, port numbers are divided into three ranges: Well Known Ports, Registered Ports, and Dynamic and/or Private Ports. The Well Known Ports are those from 0 through 1023 and SHOULD NOT be used. Registered Ports are those from 1024 through 49151 should also be avoided too. Dynamic and/or Private Ports are those from 49152 through 65535 and can be used. Though nothing is stopping you from using reserved port numbers, our suggestion may help avoid technical issues with port allocation in the future.

Step 3: Switch over to the new port by restarting SSH.

/etc/init.d/ssh restart

Step 4: Verify SSH is listening on the new port by connecting to it. Note how the port number now needs to be declared.

ssh username@hostname.com -p 50683

Note: If you have a firewall installed on your system, make sure to  also open the port you want to use in the firewall to prevent yourself from getting locked out of the system.

Get memory usage of a process under Linux

Finding out the memory usage of a process under Linux can be a bit confusing. That is because Linux doesn’t have one number for a process’ memory usage.

It has a bunch of different figures for a process’ memory usage! The different numbers include or exclude the virtual memory or swap usage that does not count towards a process’ physical memory usage.

The number that tells you the physical memory or main memory usage is the resident set size. You can see the resident set size for a process using this command:

ps -C <process name> -O rss

For example on CentOS you can find out the process size of the Apache processes using this command:

ps -C httpd -O rss

The output should be like this:

  PID   RSS S TTY          TIME COMMAND
 7409 23740 S ?        00:00:00 /usr/sbin/httpd
 7416  5484 S ?        00:00:00 /usr/sbin/httpd
 7903  8580 S ?        00:00:30 /usr/sbin/httpd

The RSS column tells you the amount of non-swaped physical memory the process is using in KB. At least that is the theory. Often parts of physical memory are shared between processes so the numbers don’t always add up. In fact most processes use shared libraries that are only loaded into memory once and shared among all processes that use them. To find out the amount of non-shared memory a process is using you use this command:

ps -C <process-name> -O size

You will get output like this:

PID    SZ S TTY          TIME COMMAND
 7804 20964 S ?        00:00:02 /usr/sbin/httpd
 7835 12692 S ?        00:00:00 /usr/sbin/httpd
 7903  3024 S ?        00:00:30 /usr/sbin/httpd

The SZ column tells you the amount of private memory a process is using in kilobytes.

But how to find out exactly how much RAM a set of processes like Apache are using? Well the answer is that it’s complicated. I try to estimate memory usage using this script:

#!/bin/bash
ps -C $1 -O rss | gawk '{ count ++; sum += $2 }; END {count --; print "Number of processes =",count; print "Memory usage per process =",sum/1024/count, "MB"; print "Total memory usage =", sum/1024, "MB" ;};'

Save it as psmem.sh and run it like this:

[admin@serve3 ~]$ psmem httpd
Number of processes = 3
Memory usage per process = 9.83464 MB
Total memory usage = 29.5039 MB

Source: http://abdussamad.com/

Information on latest downtime [ 10/07/2012 - 10/14/2012 ]

Oct 7, 2012

Nagstart po ang problem last October 7. Naging unresponsive yun server ng PINOYDEN inakala namin na may problem lamang yun database or network ng server. So ang ginawa ko triny irestart ang server pero walang effect so inisip ko baka may problema lang ang network connection ng server kaya wala magawa kundi maghintay sa reply ng tech support.

After ilang oras bumalik na ulit yun server. Pero nagloko ulit.

---

Oct 8, 2012

Online na ulit yun server pero hindi pa accessible dahil nag full backup kami ng database. After ilang oras online na ulit yun server.

We checked the server and found out that we are getting a lot of port scan attempts on our server. Di ko pinansin dahil we normally get a lot of these on some of our servers. Pero naghihinala na ako dahil marami kaming IP addresses na nadetect na galing sa bayantel 210.4.15.xxx. 

Should I follow up the IP addresses to find out who is using them at the time when the UDP packets were sent? Baka pwede natin makilala kung sino sya [/i] Nope that would be just a waste of time. I am currently working on DroidVPN with the new design at that time so nagfofocus nalang muna ako sa baging GUI ng app kaysa mag investigate

---

Oct 13, 2012

Saturday around 9PM nagkasunog sa kapitbahay. F**K malaki na ang sunog Kailangan isecure ang laptop at ang aking potpot nagpanic ng konti pero buti nalang mabilis ang mga bombero at napatay ang apoy bago pa matupok ang aming tinitirhan

Naubos ang energy namin sa sunog kaya di ko nalaman na down na pala ang forum siguro yun ay nagstart ng 12:00 AM

Kinaumagahan pag open ng computer yun down ulit ang server :slap: last time ang iniisip namin ay nagkaproblema lang ang network ng datacenter pero sa ngayun ang ibang servers namin sa same datacenter ay online so I am sure na may nangyayaring hindi maganda sa server.

I temporarily redirected the server to a temporary server to show the maintenance page. That's when I found out that the server was being DDoSed because the temporary server started to become sluggish after redirecting our domain.

Our server's IP was then null routed so wala na kaming magawa kundi hintayin na ireinstate yung IP addresses.

Niredirect nalang muna namin sa facebook/blog yun domain para malaman ng ibang users kung ano ang nangyayari.

---

Oct 14, 2012

After ma ibalik ang IP inayos na ulit namin ang server at ayun nga nagbalik na ang PINOYDEN

---

So in short naging offline ang server ng ilang araw dahil may DDoS attack sa server ng PD. Wala naman tayong malaking fund para ipambili ng Anti-DDoS appliance or mag avail ng Anti-DDoS Service.

We were not able to use cloudflare's service since the main server is already null routed. Aside from that cloudflare cannot actually offer you true DDoS protection especially if the attacker knows what hes doing. It is actually very easy to bypass cloudflare

Hindi namin alam kung bakit nila tinarget ang PD. Para sa akin karagdagang experience ang nangyari pasalamat nalang tayo at nandito parin tayo

DDoS attacks on PinoyDen Server

Update lang po tungkol sa ating server. Wala po nasira sa server naooverload lang yun network ng server dahil sa sobrang dami ng data na dumarating.

Quote from DC's Tech Support

"The actual traffic coming in was about 3+ GBPS. in which 800 was going to your port and the remainder was being hit by our routers causing severe performance issues for customers in that cabinet."

This is a very large attack that is why wala tayong magawa since mahal po ang mga Anti-DDoS Appliances or Anti-DDoS services. Pero rest assured that I am doing everything I can to restore the website. Sa ngayun po aagaayos na po kami ng temporary server at sana ay kayanin nya yung ganun kalaking traffic.

Posible po na magtagal bago bumalik ang PD dahil sobrang dami ng files sa server at medyo matagal maglipat ng files. Maguupdate narin tayo ng forum so magiging back to default theme ang PD at ang ibang features ay hindi pa available. But don't worry dahil walang post na mawawala.

Pinakamabilis na po siguro ang 24hours bago matapos.

Kita kita po tayo sa pagbabalik ng PD

Ang unang makikita naming online may premyo

MySQL & PHP 5.4 incompatibility

I just upgraded from PHP 5.3 to 5.4 but now my scripts are showing this error

The server requested authentication method unknown to the client

 
So I run a search and found this

MySQL introduced longer password hashes in (i think) version 4.1, and your server probably still uses them (check for 16 byte password hashes in your mysql user table). Newer versions use longer password hashes. Your server supports both, but your client (php) seems to support only new ones in this version (and on).

The solution

First check if "old_passwords=1" is in your my.cnf. Remove that first then restart MySQL.

Then login to your MySQL server and run the command below.

SET PASSWORD FOR 'username'@'hostmask' = PASSWORD('thepassword')