Rcom tips

search xml output

 * http://us.qa3search.g3.reuters.com/rsearch/xmlOutput.do?q=apple&start=0&site=JP&sort=date:D:S:d1

check article contain video
mysql> select PACKAGE_ID FROM NEWSGROUP WHERE UID='USKCN0ST22V20151105' \G; *************************** 1. row *************************** PACKAGE_ID: 940151762
 * article/2015/11/05/us-usa-fed-yellen-idUSKCN0ST22V20151105
 * find article in NEWSGROUP table

mysql> SELECT * FROM PACKAGENEWS WHERE ID=940151762 \G; *************************** 1. row *************************** ID: 940151762 DUID: tag:reuters.com,2015:newsml_KCN0ST22U:20
 * find in PACKAGENEWS table


 * use duid get xml from agency webservice
 * http://10.90.39.223/mc-tools/mc.html
 * tag:reuters.com,2015:newsml_KCN0ST22U:20

      113294656
 * find "video" like below


 * the 113294656, and last part of tag:yospace.com,0000:rss_113294656, is yospace id

mysql> SELECT * FROM FEEDVIDEO WHERE FEED_STORY_ID=113294656 \G;
 * find in FEEDVIDEO table

admin tomcat cluster

 * admin tomcat config cluster info, meet a problem tomcat can't start. The cluster should config point to self ip

clear apache disk cache
/rt/apache/cache
 * clear apache disk cache, you may find apache cache in folder like:

akamai staging env
How to test using Akamai's Staging Network

Applies To

Description

How do I test using Akamai's Staging Network?

Akamai's Staging Network allows Akamai's customers to test configuration changes prior to going live on the Production Network. How exactly does one test from Staging?

Testing involves two steps: first, find a Staging IP, and second, spoof your domain to this IP and test whether or not your site or application behaves as expected.

1) Here is how you find an Akamai Staging IP. Say, Akamai is serving your digital property www.foo.com. Typically, this digital property would be cnamed to an Edge Hostname such as www.foo.com.edgesuite.net or www.foo.com.edgekey.net.

(Note, if this is a new property that has not been Akamaized yet, this cname may not be in place yet. In this case, you will have to create an Edge Hostname to which you will be cnaming prior to running this test.)

$ dig www.foo.com ;; ANSWER SECTION: www.foo.com. 7200 IN CNAME www.foo.com.edgesuite.net. www.foo.com.edgesuite.net. 1359 IN CNAME a1234.g.akamai.net.

To find a Staging IP, add "-staging" to your Edge Hostname as follows:

www.foo.com.edgesuite-staging.net or www.foo.com.edgekey-staging.net

Resolve the newly obtained hostname to find a Staging IP.

$ dig www.foo.com.edgesuite-staging.net ;; ANSWER SECTION: www.foo.com.edgesuite-staging.net. 1264 IN CNAME a1234.g.akamai-staging.net. a1234.g.akamai-staging.net. 20 IN A 165.254.92.146 a1234.g.akamai-staging.net. 20 IN A 165.254.92.138

The "-staging" Edge Hostnames are created automatically when the Production Edge Hostnames "www.foo.com.edgesuite.net" or "www.foo.com.edgekey.net" are created. As with production Edge Hostname, the "-staging" Edge Hostnames resolve to optimal Staging IPs.

2) Once you have found a Staging IP, spoof www.foo.com to this IP and test your site or application as you would in Production. You can spoof by entering this domain and IP in your hosts file, or use testing tools such as curl, wfetch, or Akamai's homegrown tool EdgeSuite Booster available to download under Tools on the Luna Control Center.

Testing on Staging is strongly recommended when you make your own changes to your configuration. It is required when a change is implemented by Akamai.

You may also want to make sure that once you have correctly spoofed to the staging network, that your test requests are in fact going through it. KB Article 4856 explains these validation steps in further detail.

If you are trying to perform functional or A/B testing for users in China on a property subscribed to China CDN Service, KB 4651 describes special considerations for that testing scenario.

build rcom
>sudo make switch app=rcom-betaus >sudo make context app=rcom-${edition} CATALINA_BASE=/rt/tomcat/cms

rcom add new edition

 * http://10.90.22.6/reutersmediawiki/index.php/BjMediaDev/Staff/Liang_Zhang/Tips/CreateRcomModule
 * https://thehub.thomsonreuters.com/docs/DOC-904393

check article main text in newsdb
mysql> SELECT * FROM NEWSGROUP WHERE UID='USKBN0FL07220140716' \G; UID: USKBN0FL07220140716 PACKAGE_ID: 833067408 ...         MAIN_TEXT_ID: 935512347 MAIN_PICTURE_ID: 935512348 mysql> SELECT * FROM BASICPUBLICATION WHERE ID="935512347" \G; ID: 935512347 FILE_NAME: tag:reuters.com,2014:newsml_KBN0FL072:1 REVISION_DATE: 2014-07-16 03:11:43 HEADLINE: Exclusive: U.N. monitors allege 'conspiracy' to divert Somali assets ...
 * find uid in url http://www.reuters.com/article/2014/07/15/us-usa-immigration-idUSKBN0FK29N20140715, the uid is "USKBN0FK29N20140715"
 * use uid search NEWSGROUP table
 * use main_text_id search BASICPUBLICATION table

get dns info dnsinfo_
spotlight/install-cron.sh source /rt/baselayout/all.sh evalalltxtrecords echo $dnsinfo_g3mds_service_host

activemq config in app box
less /rt/properties/quoteserver-static.properties jms.queue.mdqueue-8.serverUrls=tcp://1
 * find mdsloader-01, /rt/properties
 * 1) produsx-app-01

telent host port ^] >telnet quit

find page from url
select * from site_map_url where domain_id=7 and url='/search' \G;
 * https://thehub.thomsonreuters.com/docs/DOC-831648 - cms page render workflow
 * find url /search, in cms db, site_map_url table

find no cache parameter
web box /rt/apache/betaus/qs.txt
 * http://10.90.22.6/reutersmediawiki/index.php/Apache_caching_query_string_params

[distuser@qa3us-cache-01 cache-server]$ sudo /etc/init.d/rcomcsd start
 * restart webbox apache will update qs.txt from db, and it's need cache server start

qa env proxy info (ftp box)
[distuser@qa7-ftp-01 conf.d]$ pwd /opt/apache22/conf/conf.d/proxy.conf [distuser@qa7-ftp-01 proxy]$ pwd /opt/apache22/conf/conf.d/proxy [distuser@qa7-ftp-01 proxy]$ grep -ir 23375 * rcom-vhosts.conf: rcom-vhosts.conf: ProxyRemote * http://10.90.108.116:23375

catch up
http://10.90.1.7/cgi-bin/trac.cgi/wiki/NtransCatchup
 * on bdnloader box

/rt/apps/ntrans-reader-server/server/catchup_script.sh shell>./catchup_script.sh MM/DD/YY channleId [ channleId2 ...] shell>./catchup_script.sh 11/17/14 40050030

telnet localhost 23012 LSREQ 1.0 SETMISSINGRANGE 40050030 1415836800000 1416212395000 ctrl + ] quit

add slideshow to qa6 env
/rt/shared/rmpd/reuters_rps/archives/
 * copy slide show tar file from qa2, on qa2-ftp-01 box

such as, /rt/shared/rmpd/reuters_rps/archives/2015-04-03/2015-04-03-qa.tar

/rt/shared/nas01/reuters_rps untar the file sudo tar xopf 2015-04-03-qa.tar
 * copy this file to qa6-ftp-01 box, on folder

SELECT * FROM news.CATEGORY where edition='us' and name='rcomus_all'
 * then the slideshow file will be dealwith by cron job shell, be sure ntrans will read this slideshow channel
 * find slideshow channle id by name

video channel type
MrssFeedVideo
 * for slideshow there is only one channel in category table

SELECT * FROM news.CATEGORY where edition='US' and package_type='MrssFeedVideo'

find related article
SELECT package_id, count(package_id) from PACKAGEMAP where role='sidebar' group by package_id order by package_id desc limit 1000;
 * in PACKAGEMAP table find ROLE='Sidebar' package

SELECT * FROM news.PACKAGENEWS where id = 894653316
 * find packageid which have more sidebar contents, use package_id find in PACKAGENEWS

article image location
/rt/shared/nas02/view/2015/03/30/106156436-1.JPG
 * image location, the "view" folder is large image
 * the "thumb" is small image, but seldom use.


 * the admin box has write permission to /rt/shared/nas02, other box can't

the param, i=106156436 , or r=106156436-1 is image name the d=20150330, is image folder
 * the video image url like: http://10.35.60.120/resources/r/?d=20150330&i=106156436&w=161x90&r=106156436-1&t=2

clear qa activemq content
clear sudo /etc/init.d/activemq stop sudo rm -rf /data02/rt/store/activemq/kahadb/* sudo /etc/init.d/activemq start
 * Activemq size too large, use entire /data02 disk

check marketdata company description with upstream
SELECT q.repno, q.ric, q.issue_PI, em.mdna_exchange, q.issuerid, q.companyname, q.country, q.primaryflag, q.type, q.category, q.tickersymbol, em.exchangename, eex.favored, q.issuer_PI FROM quoteview q JOIN exchangemap em ON em.mdna_exchange = q.exchange JOIN exchange_edition_xref eex ON eex.exchangeid = em.id JOIN edition ed ON eex.editionid = ed.id WHERE (q.ric = 'YUUZ.SI' and q.ric  collate utf8_bin = 'YUUZ.SI') AND ed.id = 21 ORDER BY eex.favored DESC, eex.ordering ASC, q.primaryflag DESC;
 * com.reuters.rcom.dataloaders.marketdata.CompanyOverviewDataLoader
 * find repo number with ric (on mdsbulkmaster-01, marketdaba db)

mysql> select * from quoteview where ric='yuuz.si';
 * another sql for find repono, may only use quoteview table;

mysql> select repno,  productionDate, numEmployees, numEmployeesLastUpdated, totalSharesOut, commonShareholders,   incorporatedState,  incorporatedYear, publicSince, streetLine1, streetLine2, streetLine3, streetLine4,   streetLine5, city, stateRegion, postalCode, country, contactPersonName, contactPersonTitle, email,   mainPhone, mainFax, contactPhone, businessSummary, equityComposition, analystFootnotes,   financialSummary from companygeninfo where repno = 'ABAA6' \G;
 * find companygeninfo

appdev@produsx-mdsrt-01 mysql> select * from bulk_job_name_value where value='companygeninfo'; +++ | name          | value          | +++ | tables_geninfo | companygeninfo | +++
 * on mdsrt-01 find which job update table companygeninfo (rtmdb.bulk_job_name_value)

[appdev@produsx-mdsbulkmaster-01 geninfo]$ pwd /rt/shared/nasmds/marketdata/sqlarchive/full/geninfo or /rt/shared/nasmds/marketdata/sqlarchive/incremental/geninfo grep -ir "ABAA6" *
 * find the job on mdsbulkmaster-01 ,

qa2 mysql vip can't access
[distuser@qa2-admin-01 ~]$ host 10.90.108.50 50.108.90.10.in-addr.arpa domain name pointer qa2-admindb-vip.g3.reuters.com
 * when meet qa2 mysql vip can't access

[distuser@qa2-trtaupdb-01 ~]$ sudo /etc/init.d/vip_QA2ADMIN_up
 * start vip shell

stage apache rewrite
svn co http://10.90.0.160/svn/webstack/releases/CKB-9521/httpd-webrail/ httpd-webrail cd httpd-webrail/rewrites make clean stage-kanban env=all remoteuser=qastage kanban_branch=[releases|features]
 * on staging box


 * find new version-tag http://10.90.0.160/svn/webstack/tags/

qatage/qastage

sudo chown -R distuser:rcom /rt/staging/qa/apache-rewrites/Release-1.0-287 cd /rt/staging/qa/apache-rewrites/Release-1.0-287 tar xzvf apache-rewrites-1.0-287.tgz
 * on staging box

export env=qa2 cd /rt/staging/qa/apache-rewrites/Release-1.0-287 make dist env=${env} remoteuser=distuser servers="${env}-web-01 ${env}-web-02" make install env=${env} remoteuser=distuser servers="${env}-web-01 ${env}-web-02" make restart-apache env=${env} remoteuser=distuser servers="${env}-web-01 ${env}-web-02"
 * deploy apache rewrie


 * detail reference CKB-9521

mdsbulk synchronization
1. In realtime db, check job name mysql> SELECT * FROM `bulk_job_name_value` WHERE `value` = 'quoteview' LIMIT 0,1000; +-+---+ | name               | value     | +-+---+ | tables_symbology_V2 | quoteview | +-+---+ 1 row in set (0.05 sec)

2. in mdsbulk-01 Stop crond and running job [distuser@qa2-mdsbulkmaster-01 ~]$ sudo /etc/init.d/crond stop

Run job in mdsbulk-01 sudo /rt/apps/md-jobs/bin/stagejobrequest.sh 'f' symbology_V2 Restart crond once the synchronization is done [distuser@qa2-mdsbulkmaster-01 ~]$ sudo /etc/init.d/crond start

rcom elastic search docs

 * https://thehub.thomsonreuters.com/docs/DOC-1069931 - Overview of Elasticsearch in Rcom
 * https://thehub.thomsonreuters.com/docs/DOC-990922 - Prepare dev evnvironment of Elasticsearch
 * https://thehub.thomsonreuters.com/docs/DOC-1643546 - How to reindex/catchup data for Elasticsearch
 * https://thehub.thomsonreuters.com/docs/DOC-1108802 - Elasticsearch Servers in RCOM
 * https://thehub.thomsonreuters.com/docs/DOC-1069931 - Overview of Elasticsearch in Reuters.com

Journalist profile page

 * https://thehub.thomsonreuters.com/docs/DOC-1646442 - Overview of Journalist profile page on Reuters.com