$``i6BSrSSKrSSKJr SSKrSSKrSSKJrJr SSK r SSK J r SSK r "SS5r\S:Xa\"S 5 \"S 5 \"5r\R$S Sr\"S \35 \R)\5r\(a\"S 5 \"S\S35 \"S\S35 \"S\"\S535 \"S\"\S535 Sr\R1\\5r\"S\S35 \"S\SS35 \"S\S35 \"S\S35 ggg) zW Competitor Content Analyzer Scrapes and analyzes competitor websites for SEO insights N) BeautifulSoup)urljoinurlparse)Counterc\rSrSrSSjrSSjrSrSrSrSr Sr S r S r S r S rS rSrSrSSjrSrSrg)CompetitorAnalyzerc[US5n[R"U5UlSSS5 [R "5UlUR RRSS05 g!,(df  NP=f)zX Initialize competitor analyzer Args: config_file: Path to configuration JSON file rNz User-Agentz$>t$D 2248!006"&":":4"E"&":":4"EdmmE23!%!1!1$!7 G%( (<(B(B(D$EGL !N  -cU"QC8 9 sDD E*EEcrURS5nU(aUR5R5$S$)zExtract page titler/)findget_textstrip)rrG title_tags rr;CompetitorAnalyzer._get_title^s/IIg& /8y!!#))+@b@r$cxURSSS0S9nU(a URSS5R5$S$)zExtract meta descriptionmetaname descriptionattrsr9rL)rMrrO)rrGrSs rr<(CompetitorAnalyzer._get_meta_descriptioncs:yy '>y?26txx 2&,,.>B>r$c/n[SS5HNnURSU35H3nURUUR5R 5S.5 M5 MP U$)z#Extract all headings with hierarchyh)leveltext)rangerBappendrNrO)rrGr1r]headings rr=$CompetitorAnalyzer._extract_headingshs`1a[E==1UG5"#,,.446!6! r$cURS5=(d) URS5=(d URS5nU(a*URSSS9n[R"SSU5nU$g) zExtract main body textmainarticlebody T) separatorrOz\s+rL)rMrNresub)rrG main_contentr^s rr>!CompetitorAnalyzer._get_body_textss`yy(UDIIi,@UDIIfDU ((3d(CD66&#t,DKr$c[U5Rn/nURSSS9H?nUSn[X&5n[U5RU:XdM.UR U5 MA [ [ U55$)zExtract internal linksaThrefrp)rnetlocrBrr`listset)rrGbase_urldomainr4linkrpfull_urls rr?&CompetitorAnalyzer._get_internal_linksst(#**MM#DM1D# UH%oSS:XdM TUSR5;v M' g7f)r]rZr^N)lower).0r\ keyword_lowers r ?CompetitorAnalyzer.analyze_content_structure..s4M%8gJ!O@]aioo.??%8s 00r1r. keyword_countkeyword_densitytitle_has_keywordmeta_has_keywordh1_has_keywordh1_countr]rZh2_counth3_counttotal_headingsr4r5r6 has_schemar7keyword_variations)rcountany_find_keyword_variationsroundrA) rr9r text_lowerrrrrrrr\analysisrs @ranalyze_content_structure,CompetitorAnalyzer.analyze_content_structurese[)//1   #((7 KRS_K`cdKd=<+@@3Fjk*WW-=-C-C-EE(G4F,G,M,M,OOM%,Z%8MM":::}U 75> WW%  ',/  ]  u_a8  !2    0  n   (;O(;1zQQ(;OP   (;O(;1zQQ(;OP   (;O(;1zQQ(;OP  c'*"56  c'*:";<  c'*:";<  gh' '/2! " !"4# $  +% *POOs$" F 2F F F 6 F# F# c ^UR5n[5nUR5n[[U55Hrn[US-[ US-[U5S-55HCnSR XVU5m[ U4SjU55(dM2URT5 ME Mt [U5SS$)z&Find variations of the keyword in textrZrgc3,># UH oT;v M g7f)N)rkwphrases rr>CompetitorAnalyzer._find_keyword_variations..s#J$$r$cUR5nUR5n[UR55nURU5nUS:XagXe- S-$)z$Calculate keyword density percentagerr)rrArDr)rr^r rrr3rs rcalculate_keyword_density,CompetitorAnalyzer.calculate_keyword_densitysPZZ\   & "((7 ?*c11r$c hU(d0$UVs/sH o"cMUPM nnU(d0$[U5[SU55[U5- [SU55[U5- [SU55[U5- [SU55[U5- [SU55[U5- S-[SU55[U5- S-[SU55[U5- S-[S U55[S U55US . nU$s snf) z~ Compare multiple competitor analyses Args: analyses: List of analysis dictionaries Returns: Comparison summary c3*# UH oSv M g7fr3Nrrrns rr9CompetitorAnalyzer.compare_competitors.. !J>aL/>c3*# UH oSv M g7f)rNrrs rrr s&T^):';^rc3*# UH oSv M g7f)rNrrs rrr sL^"2 3^rc3*# UH oSv M g7f)r4Nrrs rrr s%R>a(8&9>rc3<# UHoS(dMSv M g7f)rrZNrrs rrrs'\>aGZE[>  rc3<# UHoS(dMSv M g7f)rrZNrrs rrrs$V1DTBUQQrc3<# UHoS(dMSv M g7f)rrZNrrs rrrs#Q~!<AA~rc3*# UH oSv M g7frrrs rrrrrc3*# UH oSv M g7frrrs rrrrr) total_analyzedavg_word_countavg_keyword_density avg_headingsavg_internal_linkskeyword_in_title_pctkeyword_in_h1_pctschema_usage_pctmin_word_countmax_word_countr)rAsumrmax)ranalysesrnvalid_analyses comparisons rcompare_competitors&CompetitorAnalyzer.compare_competitorss,I%-?X!X?I".1!!J>!JJSQ_M``#&&T^&T#TWZ[iWj#jL^LLsSaObb"%%R>%R"RUXYgUh"h$''\>'\$\_bcq_r$rux$x!$$V$V!VY\]kYl!lor!r ##Q~#Q QTWXfTg gjm m!!J>!JJ!!J>!JJ)  '@s D/D/c[SU35 [5nU/n/nU(a[U5U:aURS5nXc;aM.UR U5 UR U5nU(a?UR U5 USSSH"nX;dM X;dMUR U5 M$ [R"S5 U(a[U5U:aM[S[U5S35 U$) z Scrape internal pages from a website Args: base_url: Starting URL max_pages: Maximum pages to scrape Returns: List of scraped page data z Scraping internal pages from: rr4NrZu ✓ Scraped z pages) rrsrApoprrIr`timesleep) rrt max_pagesvisitedto_visit scraped_pagesr.r9rvs rscrape_internal_pages(CompetitorAnalyzer.scrape_internal_pagess 0 ;<%: 3}- 9,,q/C~ KK ..s3G$$W-$$45bq9D*t/C -: JJqM#3}- 9& s=12&9:r$cSRSU55n[R"SUR55n/n[ SS5HJn[ [ U5U- S-5H)nSRX6Xe-5nUR U5 M+ ML [U5n1Skn URS5VV s/sHupzXy;dM U S:dMXz4PM n nn U $s sn nf) z Extract all potential keywords from scraped pages Args: pages_data: List of page content dictionaries Returns: List of keyword phrases with frequency rgc3<# UHo(dM USv M g7f)r2Nr)rpages rr:CompetitorAnalyzer.extract_all_keywords..IsM*$-D-*s  z\b\w+\brrrZ>at thein theof theon theto thefor ther) rrifindallrr_rAr`r most_common) r pages_dataall_textrphrasesnrr phrase_counts stop_phrasesrfiltereds rextract_all_keywords'CompetitorAnalyzer.extract_all_keywords?s88M*MM :x~~'78q!A3u:>A-.%!#,/v&/  ( U 9F9R9RSV9W@9W 0$5:QY$VO9W@@s= C  C C )rrN)z config.json)r&)r)__name__ __module__ __qualname____firstlineno__rr"rIr;r<r=r>r?r@rCrrrrrr__static_attributes__rr$rrrsY  @&)VA ?   ) )H 3j%( 2 D%Nr$r__main__zTesting Competitor Analyzer...z<============================================================rz Scraping: u ✓ Successfully scraped!z Title: r/z Word Count: r3z Headings: r1z Internal Links: r4zconservatory roofz Keyword Analysis for 'z':z Keyword Density: r%z In Title: rz In H1: r)__doc__rbs4rr ri urllib.parserrr collectionsrpandaspdrrranalyzerrtest_urlrIr9rA test_keywordrrrr$rrsm  * MM`  z *+ &M!#H}-a0H L #$**84G +- ''*+,- w|4567 S!45678 "3w/?'@#A"BCD+ 55g|L (b9: #H->$?#@BC X&9:;<= (#34567r$