安爺ニュース/newsリンク/ ニュース拾い読み/PC備忘録

 AWStatsで,リンク元にリファラスパムアドレスが表示されないようにする

リファラスパムアクセスログ破棄用設定ファイル作成
[root@falcon21 ~]# vi /etc/httpd/conf.d/referer_spam.conf
SetEnvIfNoCase Referer (\.|/)1marketingguru\.com no_log
SetEnvIfNoCase Referer (\.|/)\.ru no_log
SetEnvIfNoCase Referer (\.|/)\.su no_log
SetEnvIfNoCase Referer (\.|/)aawebmastesr\.com no_log
SetEnvIfNoCase Referer (\.|/)admax-express\.com no_log
SetEnvIfNoCase Referer (\.|/)asianpornclips\.t35\.com no_log
SetEnvIfNoCase Referer (\.|/)ask-the-songwriter\.com no_log
SetEnvIfNoCase Referer (\.|/)assfuckdolls\.com no_log
SetEnvIfNoCase Referer (\.|/)blogspot\.com no_log
SetEnvIfNoCase Referer (\.|/)caindifest\.com no_log
SetEnvIfNoCase Referer (\.|/)cerfdom\.com no_log
SetEnvIfNoCase Referer (\.|/)com\.ar no_log
SetEnvIfNoCase Referer (\.|/)congregationdc\.com no_log
SetEnvIfNoCase Referer (\.|/)deweight\.com no_log
SetEnvIfNoCase Referer (\.|/)doktor-info\.de no_log
SetEnvIfNoCase Referer (\.|/)edringtonoptical\.com no_log
SetEnvIfNoCase Referer (\.|/)fialf\.cn no_log
SetEnvIfNoCase Referer (\.|/)formspring\.me no_log
SetEnvIfNoCase Referer (\.|/)gamehram\.ru no_log
SetEnvIfNoCase Referer (\.|/)global-wealth-management\.com no_log
SetEnvIfNoCase Referer (\.|/)haarpoint\.net no_log
SetEnvIfNoCase Referer (\.|/)harfordweb\.com no_log
SetEnvIfNoCase Referer (\.|/)heartsdesireconcerts\.com no_log
SetEnvIfNoCase Referer (\.|/)hp-www\.com no_log
SetEnvIfNoCase Referer (\.|/)ilpharm\.com no_log
SetEnvIfNoCase Referer (\.|/)inet-d\.com no_log
SetEnvIfNoCase Referer (\.|/)inspiringwebs\.com no_log
SetEnvIfNoCase Referer (\.|/)interracial.porn.c13a\.com no_log
SetEnvIfNoCase Referer (\.|/)jilview\.com no_log
SetEnvIfNoCase Referer (\.|/)joncard\.com no_log
SetEnvIfNoCase Referer (\.|/)krankenpflegedeutschland\.wordpress\.com no_log
SetEnvIfNoCase Referer (\.|/)kwiz\.biz no_log
SetEnvIfNoCase Referer (\.|/)ler-bochka\.ru no_log
SetEnvIfNoCase Referer (\.|/)lericettedicucina\.it no_log
SetEnvIfNoCase Referer (\.|/)locketech\.com no_log
SetEnvIfNoCase Referer (\.|/)mathesoninteractive\.com no_log
SetEnvIfNoCase Referer (\.|/)mckmac\.com no_log
SetEnvIfNoCase Referer (\.|/)milcard\.com no_log
SetEnvIfNoCase Referer (\.|/)mp3ll\.net no_log
SetEnvIfNoCase Referer (\.|/)mygardenlink\.com no_log
SetEnvIfNoCase Referer (\.|/)netvibes\.com no_log
SetEnvIfNoCase Referer (\.|/)neumannsouthey\.com no_log
SetEnvIfNoCase Referer (\.|/)newsexphone\.com no_log
SetEnvIfNoCase Referer (\.|/)ohab-accounting\.com no_log
SetEnvIfNoCase Referer (\.|/)pharlo\.com no_log
SetEnvIfNoCase Referer (\.|/)phenterminert\.com no_log
SetEnvIfNoCase Referer (\.|/)planettoyko\.com no_log
SetEnvIfNoCase Referer (\.|/)plumper.hardcore.e13a\.com no_log
SetEnvIfNoCase Referer (\.|/)pornstars\.name no_log
SetEnvIfNoCase Referer (\.|/)portalconference\.com no_log
SetEnvIfNoCase Referer (\.|/)pqgraphica\.com no_log
SetEnvIfNoCase Referer (\.|/)prophotorestoration\.com no_log
SetEnvIfNoCase Referer (\.|/)qlporn\.com no_log
SetEnvIfNoCase Referer (\.|/)rak-economic-dept\.com no_log
SetEnvIfNoCase Referer (\.|/)raska-kaffiya\.com no_log
SetEnvIfNoCase Referer (\.|/)removed\.com no_log
SetEnvIfNoCase Referer (\.|/)rubanke\.ru no_log
SetEnvIfNoCase Referer (\.|/)seawolfmm\.com no_log
SetEnvIfNoCase Referer (\.|/)sekwebdesign\.com no_log
SetEnvIfNoCase Referer (\.|/)senauka\.ru no_log
SetEnvIfNoCase Referer (\.|/)slouchco\.com no_log
SetEnvIfNoCase Referer (\.|/)spectronixresearch\.com no_log
SetEnvIfNoCase Referer (\.|/)spiraltouch\.com no_log
SetEnvIfNoCase Referer (\.|/)stlmortgageconsultants\.com no_log
SetEnvIfNoCase Referer (\.|/)sweetplace\.de no_log
SetEnvIfNoCase Referer (\.|/)taskflex\.com no_log
SetEnvIfNoCase Referer (\.|/)technologyevangeslist\.com no_log
SetEnvIfNoCase Referer (\.|/)thebraincell\.net no_log
SetEnvIfNoCase Referer (\.|/)treatstreet\.net no_log
SetEnvIfNoCase Referer (\.|/)tumblr\.com no_log
SetEnvIfNoCase Referer (\.|/)tuning-tech\.de no_log
SetEnvIfNoCase Referer (\.|/)usall\.de no_log
SetEnvIfNoCase Referer (\.|/)videoseighteen\.com no_log
SetEnvIfNoCase Referer (\.|/)visionecon\.com no_log
SetEnvIfNoCase Referer (\.|/)web-prose\.com no_log
SetEnvIfNoCase Referer (\.|/)wfpearsonatgmail\.com no_log
SetEnvIfNoCase Referer (\.|/)xoweight\.com no_log
SetEnvIfNoCase Referer (\.|/)asapautos\.ca no_log
SetEnvIfNoCase Referer (\.|/)outlet-tipps\.de no_log
SetEnvIfNoCase Referer (\.|/)alecso\.net no_log
SetEnvIfNoCase Referer (\.|/)wie-abnehmen\.org no_log
SetEnvIfNoCase Referer (\.|/)rentor\.de no_log
SetEnvIfNoCase Referer (\.|/)xn--gebrauchtwagenhndler-pzb\.de no_log
SetEnvIfNoCase Referer (\.|/)rdbooks\.com no_log
SetEnvIfNoCase Referer (\.|/)xn--gnstig-fliegen-gsb\.de no_log
SetEnvIfNoCase Referer (\.|/)tourextreme\.ru no_log
SetEnvIfNoCase Referer (\.|/)\.kz no_log
SetEnvIfNoCase Referer (\.|/)web-prose\.com no_log
SetEnvIfNoCase Referer (\.|/)web-prose\.com no_log
SetEnvIfNoCase Referer (\.|/)web-prose\.com no_log

--------------------------------------------------------------- 
Apacheログからリファラスパムを除去するスクリプトを作成
[root@falcon21 ~]# vi refererlogdel.sh
#!/bin/bash
for host in `cat /etc/httpd/conf.d/referer_spam.conf|awk '{print $3}'|sed -e 's/^(.*)\([^ ]*\)/\1/p' -e d`
do

grep "http://.*${host}" ${1} > /dev/null
if [ $? -eq 0 ]; then
echo "${host} `grep "http://.*${host}" ${1}|wc -l`"
sed -i "/http:\/\/.*${host}/d" ${1}
fi
done
which systemctl > /dev/null 2>&1
if [ $? -eq 0 ]; then

systemctl restart httpd
else

/etc/rc.d/init.d/httpd restart
fi


-------------------------------------------- 
WStatsログからリファラスパムを除去するスクリプトを作成
[root@falcon21 ~]# vi awstatslogchk.sh
#!/bin/bash

if [ $# -eq 0 -o ! -f /etc/awstats/awstats.${1}.conf ]; then

log=`grep ^DirData /etc/awstats/awstats.model.conf|sed -e 's/^DirData="\([^ ]*\)"$/\1/p' -e d`/awstats*.txt
else

log=`grep ^DirData /etc/awstats/awstats.${1}.conf|sed -e 's/^DirData="\([^ ]*\)"$/\1/p' -e d`/awstats*${1}.txt
fi

for host in `cat /etc/httpd/conf.d/referer_spam.conf|awk '{print $3}'|sed -e 's/^(.*)\([^ ]*\)/\1/p' -e d`
do

count=`grep "${host}" ${log} | wc -l`
if [ ${count} != "0" ]; then
sed -i "/$host/d" $log
echo "${host} ${count} deleted."
fi
done

------------------------------------------------ 
リファラスパム除去 
スパムアドレスを追加 既に、/etc/httpd/conf.d/referer_spam.conf にある登録様式を参考に  /var/log/httpd/access_logからリファラスパム削除
[root@falcon21 ~]# sh refererlogdel.sh /var/log/httpd/access_log 
AWStatsログからリファラスパム削除
[root@falcon21 ~]# sh awstatslogchk.sh

Update database 
[root@falcon21 ~]# sh awstatsinit.sh falcon21.space
Create/Update database for config "/etc/awstats/awstats.falcon21.space.conf" by AWStats version 7.7 (build 20180105)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Direct access after last parsed record (after line 475)
Jumped lines in file: 475
 Found 475 already parsed records.
Parsed lines in file: 25
 Found 2 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 23 new qualified records.
Create/Update database for config "/etc/awstats/awstats.falcon21.space.conf" by AWStats version 7.7 (build 20180105)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Direct access after last parsed record (after line 500)
Jumped lines in file: 500
 Found 500 already parsed records.
Parsed lines in file: 0
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 0 new qualified records.

***********************************************************