Endeca Controller - start and stop Endeca Server - Write in progress with generation

Concurrent Update error : 

ispatcherThread.java:178)
**** Error      Thu Jun 19 08:13:03 PDT 2014    1403190783340   /atg/commerce/endeca/index/CategoryToDimensionOutputConfig
**** info       Thu Jun 19 08:13:03 PDT 2014    1403190783355   /atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin    Indexing process cancelled, Endeca says: atg.repository.search.indexing.IndexingException: com.endeca.itl.recordstore.ConcurrentWriteException: Write in progress with generation 42
**** Error      Thu Jun 19 08:13:03 PDT 2014    1403190783357   /atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin    ---     atg.repository.search.indexing.IndexingException: com.endeca.itl.recordstore.ConcurrentWriteException: Write in progress with generation 42

 The below script can be use to start and stop the three Endeca Services.
Also clear the logs before restarting.

##############
#!/bin/bash


########################################################
# #     Endeca Clean and Start Scripts               ###
# #                                                  ###
# #     Author : Anilal KG                           ###
# #                                                  ###
# #     Version  : 0.1                               ###
########################################################


SCRIPT=endeca
ENDECA_INSTALL_BASE=/path/to/endeca

PS_BIN=${ENDECA_INSTALL_BASE}/PlatformServices/6.1.4/tools/server/bin
TF_BIN=${ENDECA_INSTALL_BASE}/ToolsAndFrameworks/3.1.2/server/bin
CAS_BIN=${ENDECA_INSTALL_BASE}/CAS/3.1.2.1/bin
APP_LOG=${ENDECA_INSTALL_BASE}/apps/$1/logs/  ## path to endeca apps logs

if [ -z "$1" ]
    then
         echo ""
            echo "Usage: "
            echo ""
            echo "  sh endecacontroller.sh  <appname> "
            exit
fi

if [ ! -d "$APP_LOG" ]; then
    printf "\n The ENDECA application -- $1 -- does not exists .Please check once again \n"
    exit
fi

echo ""
echo "Stopping CAS  ".${CAS_BIN}
${CAS_BIN}/cas-service-shutdown.sh


echo ""
echo "Stopping Tools And Frameworks  ".${TF_BIN}
${TF_BIN}/shutdown.sh


##
echo ""
echo "Stopping PlatformServices  ".${PS_BIN}
${PS_BIN}/shutdown.sh 


## clearing logs
echo ""
echo "Clearing the application logs .."
echo $APP_LOG
rm -rf $APP_LOG/*.*



echo ""
echo "Starting Platform Services  "
${PS_BIN}/startup.sh 


echo ""
echo "Starting Tools And Frameworks  "
${TF_BIN}/startup.sh


echo ""
echo "Starting CAS  "
${CAS_BIN}/cas-service.sh &


echo ""
echo ""
echo "Status of the ports listening -double check manually --Listenting for ports..."
echo ""
echo ""

netstat -plnt | grep :::8


##############

SQL Query to find the table name which contains specific data

Requirement :  when you don't know what table or what column in your schema have the data.

This example, generates the table name / column name which contains the Data - 'samsung';

Thanks to Mohan for sending this Query. Worked fine for me hence posting .

######################

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\ag4063>sqlplus /nolog

SQL*Plus: Release 11.2.0.1.0 Production on Thu Jun 12 18:22:49 2014

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

SQL> connect sid_contenta
Enter password:
Connected.
SQL>

#####################################

spool outFile.txt

set serveroutput ON size 1000000

declare
srch_string varchar2(100) := 'samsung';
str varchar2(4000);
PROCEDURE get_column_list(tabname in varchar2) IS
begin
for c1 in(select column_name from user_tab_columns where table_name=tabname and data_type like '%CHAR%')
LOOP
str := str || 'upper('||c1.column_name||') like upper(''%'||srch_string||'%'') or ';
END LOOP;
str := rtrim(str, 'or ');
END get_column_list;
begin
for c2 in(select table_name from user_tables) LOOP
str := 'select * from '||c2.table_name||' where ';
get_column_list(c2.table_name);
dbms_output.put_line(str||';');
END LOOP;
end;
/

################################################
spool off
###############################################

Version issues with DSP Taglib

Error Log:

1398356350603               /atg/registry/webappregistry/ServletContextWebAppRegistry       
The version of the DSP Taglib from the web-app "BCC" found
within "zip:C:/ATG/ATG10.2/EAR/BCC.ear/atglib/_DAS_slib_sclasses.jar!/"
did not match the expected version. Please update all copies of the
 DSP Taglib with the version in $DYNAMO_HOME/DAS/taglib/dspjspTaglib/1.0/lib/
and re-assemble your application. Enable loggingDebug on this component
and restart to see which resource differs.


Fix: To fix above issue, replace taglibs of BCC/j2ee-apps/BCCWeb/BCCWeb.war/WEB-INF/lib from C:\ATG\ATG10.2\DAS\taglib\dspjspTaglib\1.0

Note: This fix I am yet to test it in my local. Please try and update if it works.

Intergrating ATG-Endeca for Indexing

Note: This is with Version ATG 10.2 onwards Only


Step : 1

You have to have your below component configured to use your PCOC file like below

/atg/commerce/search/ProductCatalogOutputConfig/

definitionFile=/atg/commerce/endeca/index/product-sku-output-config.xml

Sample file Attached:

Step : 2

Your combined definition file should be a union of below xmls(OOTB+Custom)
Source files   

\config\config.jar/atg/commerce/endeca/index/product-sku-output-config.xml
\DCS\Endeca\Index\config\config.jar/atg/commerce/endeca/index/product-sku-output-config.xml
\DCS\Endeca\Index\SKUIndexing\config\config.jar/atg/commerce/endeca/index/product-sku-output-config.xml


Step : 3

Build level changes : Add the below modules in your ANT build.properties file
for eg:
my.ear.modules=DafEar.Admin,DCS.Endeca.Index.Versioned,DCS.Endeca.Index,DAF.Endeca.Index,DAF.Endeca.Assembler,DCS.Endeca.Index.SKUIndexing

Verification :  Once your build is up, you should be able to see this componenet via dyn/admin
http://mylocalserver:7001/dyn/admin/nucleus/atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin/

Step : 4

Make these config layer changes.Override bases on your environments.

/atg/endeca/ApplicationConfiguration
baseApplicationName: myendecaapp
keyToApplicationName: default=myendecaapp,en=myendecaapp
workbenchHostName: myendeca.envr.server.name.com
workbenchPort: 8006

/atg/endeca/index/IndexingApplicationConfiguration
CAS Hostname: myendeca.envr.server.name.com
CAS Port: 8500

EAC Hostname: myendeca.envr.server.name.com
EAC Port: 8888

/atg/endeca/assembler/AssemblerApplicationConfiguration
currentMdexHostName: myendeca.envr.server.name.com
currentMdexPort: 16000

defaultMdexHostName: myendeca.envr.server.name.com
defaultMdexPort: 16000
applicationKeyToMdexHostAndPort: en= myendeca.envr.server.name.com:16000


Step 5: (Optional)

If you are setting up your endeca indexing from your BCC instance, you need to override the below component

Set the OnPubServer flag to false for the following components

/atg/commerce/search/ProductCatalogOutputConfig
/atg/commerce/endeca/index/CategoryToDimensionOutputConfig


Step 6:

Triggering Indexing from Dyn Admin:

Navigate : http://mylocalserver:7001/dyn/admin/nucleus/atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin/

Trigger the BaseLine Index Button.Watch your ATG logs for issues.

#################################
Reference  : http://docs.oracle.com/cd/E23095_01/Platform.93/ATGSearchIntegration/html/s0202specifyingthedefinitionfileandth01.html

How to set up your local BCC.

How to set up your local BCC.

I thought to update here what I did for setting up my ATG-BCC up and running in my local box.


Steps I did:

1. Imported the db which is running, for a working instance.
2. Cleaned up the PUB schema -- refer the below links
    http://atgtech.blog.com/2013/10/15/159/

3.  delete from EPUB_COLL_WORKFLOW
     delete from EPUB_WORKFLOW_INFO
     delete from EPUB_WF_SERVER_ID
     commit;

4. Restart the server,
  4.1.1  select * from EPUB_WF_SERVER_ID   // in your pub schema
             this should show you the server id of your local machine. Important !!!!

5. Login to BCC with the credentials of the database as in step #1.

------------------------

Thinks to ensure:

There should be role assigned properly for the user "Admin"

To verify it:

Navigate to : http:/localhost:7001/dyn/admin/nucleus/atg/userprofiling/InternalProfileRepository/

<print-item item-descriptor="user" id="portal-admin" />

------ Printing item with id: portal-admin
<add-item item-descriptor="user" id="portal-admin">
  <set-property name="securityStatus"><![CDATA[EXPLICIT-SIGNIN]]></set-property>
  <set-property name="email"><![CDATA[anilal.testing@tester.com]]></set-property>
  <set-property name="password"><![CDATA[tes]]></set-property>
  <!-- rdonly   <set-property name="accessRights"><![CDATA[previewuserfull,seotagsfull,mediaread,sitecatalogread,sitecatalogfull,promotionsfull,tgtsgmtfull,storesfull,siteadminfull,pricelistsfull,siteadminread,externaluserfull,mediafull,storecontentread,internaluserfull]]></set-property>  -->

  <set-property name="roles"><![CDATA[internalUserAdministrationRole,epubSuperAdmin,merchandiserRole,promotionsAdministratorRole,portal-admin,personalizationRole,siteAdminRole]]></set-property>
  <set-property name="login"><![CDATA[admin]]></set-property>

</add-item>


Make sure that you have these roles set up.
epubSuperAdmin -- should exists for the user admin for the property "Roles"