Python BeautifulSoup extracting the text right after a particular tag

I'm trying to extract information from a webpage using beautifulsoup and python. I want to extract the information right below a particular tag. To know if its the right tag I would like to do a comparison of its text and then extract the text in the next immediate tag. Say for example, if the following is a part of an HTML page-source,

<div class="row">
    ::before
    <div class="four columns">
        <p class="title">Procurement type</p>
        <p class="data strong">Services</p>
    </div>
  <div class="four columns">
      <p class="title">Reference</p>
      <p class="data strong">ANAJSKJD23423-Commission</p>
  </div>
  <div class="four columns">
      <p class="title">Funding Agency</p>
      <p class="data strong">Health Commission</p>
  </div>
  ::after
</div>
<div class="row">
    ::before
    ::after
</div>
<hr>
<div class="row">
    ::before
    <div class="twelve columns">
        <p class="title">Countries</p>
        <p class="data strong">
            <span class>Belgium</span>
            ", "
            <span class>France</span>
            ", "
            <span class>Luxembourg</span>
        </p>
        <p></p>
    </div>
    ::after
</div>

I want to check if the <p class="title"> has text value as Procurement type then I want to print out Services Similarly, if the <p class="title"> has text value as Reference then I want to print out ANAJSKJD23423-Commission and if <p class="title"> has value as Countries then print out all the countries i.e. Belgium,France,Luxembourg.

I know I can extract all the texts with <p class="data strong"> and append them to a list and later fetch all values using indexing. But the thing is, the order of the occurrence of these <p class="title> is not fixed....at some places countries could be mentioned before procurement-type. I, therefore, want to perform a check on the text values and then extract the next immediate tag's text value. I'm still new to BeautifulSoup so any help is appreciated. Thanks

You can do it many ways.Here you go.

from bs4 import BeautifulSoup
htmldata='''<div class="row">
    ::before
    <div class="four columns">
        <p class="title">Procurement type</p>
        <p class="data strong">Services</p>
    </div>
  <div class="four columns">
      <p class="title">Reference</p>
      <p class="data strong">ANAJSKJD23423-Commission</p>
  </div>
  <div class="four columns">
      <p class="title">Funding Agency</p>
      <p class="data strong">Health Commission</p>
  </div>
  ::after
</div>
<div class="row">
    ::before
    ::after
</div>
<hr>
<div class="row">
    ::before
    <div class="twelve columns">
        <p class="title">Countries</p>
        <p class="data strong">
            <span class>Belgium</span>
            ", "
            <span class>France</span>
            ", "
            <span class>Luxembourg</span>
        </p>
        <p></p>
    </div>
    ::after
</div>'''

soup=BeautifulSoup(htmldata,'html.parser')

items=soup.find_all('p', class_='title')
for item in items:
    if ('Procurement type' in item.text) or ('Reference' in item.text):
        print(item.findNext('p').text)

beautifulsoup, Imagine you have the following HTML: <div> <label>Name:</label> John Smith < /div>. And you need to locate the text "John Smith" after the label element. python - How to tell BeautifulSoup to extract the content of a specific tag as text? (without touching it) python - how to extract text within font tag using beautifulsoup; python - Extracting text between with beautifulsoup, but without next tag; python - BeautifulSoup: How to extract data after specific html tag; python - Parsing and

You can also use :contains pseudo class with bs4 4.7.1. Although I have passed as a list you can separate out each condition

from bs4 import BeautifulSoup as bs
import re

html = 'yourHTML'   
soup = bs(html, 'lxml')
items=[re.sub(r'\n\s+','', item.text.strip()) for item in soup.select('p.title:contains("Procurement type") + p, p.title:contains(Reference) + p, p.title:contains(Countries) + p')]
print(items)

Output:

Web Scraping with Beautiful Soup, Web scraping is a process of extracting specific information as Use BeautifulSoup to find the particular element from the response and extract the text . specific HTML tags which contain the data, so let's jump straight into� A BeautifulSoup object has several methods and attributes that we can use to navigate within the parsed document and extract data from it. The most used method is .find_all (): soup.find_all (name, attrs, recursive, string, limit, **kwargs) name — name of the tag; e.g. “a”, “div”, “img”.

You can add the argument to check for specific text when you use .find() or .find_all() then use .next_sibling or findNext() to grab the next tags with the content

Ie:

soup.find('p', {'class':'title'}, text = 'Procurement type')

Given:

html = '''<div class="row">
    ::before
    <div class="four columns">
        <p class="title">Procurement type</p>
        <p class="data strong">Services</p>
    </div>
  <div class="four columns">
      <p class="title">Reference</p>
      <p class="data strong">ANAJSKJD23423-Commission</p>
  </div>
  <div class="four columns">
      <p class="title">Funding Agency</p>
      <p class="data strong">Health Commission</p>
  </div>
  ::after
</div>
<div class="row">
    ::before
    ::after
</div>
<hr>
<div class="row">
    ::before
    <div class="twelve columns">
        <p class="title">Countries</p>
        <p class="data strong">
            <span class>Belgium</span>
            ", "
            <span class>France</span>
            ", "
            <span class>Luxembourg</span>
        </p>
        <p></p>
    </div>
    ::after
</div>'''

you could do something like:

from bs4 import BeautifulSoup     

soup = BeautifulSoup(html, 'html.parser')

alpha = soup.find('p', {'class':'title'}, text = 'Procurement type')
for sibling in alpha.next_siblings:
    try:
        print (sibling.text)
    except:
        continue

Output:

Services

or

ref = soup.find('p', {'class':'title'}, text = 'Reference')
for sibling in ref.next_siblings:
    try:
        print (sibling.text)
    except:
        continue

Output:

ANAJSKJD23423-Commission    

or

countries = soup.find('p', {'class':'title'}, text = 'Countries')
names = countries.findNext('p', {'class':'data strong'}).text.replace('", "','').strip().split('\n')
names = [name.strip() for name in names if not name.isspace()]

for country in names:
    print (country)

Output:

Belgium
France
Luxembourg

Python BeautifulSoup extracting the text right after a particular tag, I'm trying to extract information from a webpage using beautifulsoup and python. I want to extract the information right below a particular tag. To know if its the� Beautiful Soup extracting text without tags I don't want the text between the tags within the <p> tags. I do want the text that isn't in a tag between the <p> tags.

Beautiful Soup Documentation — Beautiful Soup 4.4.0 documentation, One common task is extracting all the URLs found within a page's <a> tags: Make sure you use the right version of pip or easy_install for your Python version (these Beautiful Soup uses the NavigableString class to contain these bits of text: It tells Beautiful Soup to stop gathering results after it's found a certain number. Web scraping is harvesting or extracting desired information from a webpage. Scraping using BeautifulSoup. For web scraping we are going to use the very popular Python library called BeautifulSoup. For web scraping you first need to have some basic knowledge about the HTML tags. Some of the tags used in HTML are shown below.

How to scrape text from webpage using beautifulsoup python?, How to extract the text inside "p" tags which has "a" tags in it. I recently used BeautifulSoup in Python to scrape a large data set of website URLs and the following tutorial How to extract particular section from text data using NLP in Python? p#first — finds all p tags with an id of first. body p.outer-text — finds any p tags with a class of outer-text inside of a body tag. BeautifulSoup objects support searching a page via CSS

BeautifulSoup - Find the next element after a tag - Python code , Python code example 'Find the next element after a tag' for the package Find all tags with a given name in the next siblings of a tag from bs4 import BeautifulSoup html = open("small.html").read() soup = BeautifulSoup(html) Kite is a plugin for PyCharm, Atom, Vim, VSCode, Sublime Text, and IntelliJ that uses machine� BeautifulSoup. BeautifulSoup is a Python library for parsing HTML and XML documents. It is often used for web scraping. BeautifulSoup transforms a complex HTML document into a complex tree of Python objects, such as tag, navigable string, or comment.