FlowHeater Demo Video
 
FlowHeater - The Data Import / Export Specialist
 
Welcome, Guest
Username: Password: Remember me
See also the general examples available, which may answer your question: Hello World 1 / 2 / 3 , format numbers as character strings , simple If-Then-Else , complex If-Then-ElseIf , filtering and grouping data , consolidate grouped values
  • Page:
  • 1

TOPIC: URL validation

URL validation 5 months 2 weeks ago #3509

Hello, 
Just wondering if there is a quick and easy way to validate urls as part of my flowheater process of transforming a CSV file into XML.  
I have a list of 100K urls for product links that need to validate, by showing  a 200 or 404 status code.  Thank you for the help! 

Please Log in or Create an account to join the conversation.

URL validation 5 months 2 weeks ago #3510

Hi Daniel,

There are two possible solutions maybe.
  1. Using the .NET Script Heater  and the short below listed C# script. Very fast and recommended. In this case you have to move one .NET Script Heater  on the Designer and copy the script below into. Now switch to the “Advanced tab” and add the following line, like the screenshot below.
    using System.Net;
    At least you have to connect the field containing the URLs to this Heater, the output contains the HTTP status code.
  2. Using the CMD Heater  and a simple PowerShell script (geturl.ps1). In this case you have to move one CMD Heater  on the Designer. To execute PowerShell scripts you have to use “powershell.exe” for the execute parameter. For the “Parameter” part you have to use the following line like the screenshot below.
    -ExecutionPolicy bypass -File geturl.ps1 "$1"
    Note: $1 will be dynamically replaced with the value off the first input parameter! The Heater output contains the HTTP status code.
For both I´ve made an example for you, please have a look to the attachment.

C# Script to validate a URL)
public object DoWork()
{
  int statuscode = 404;
  
  // get first input parameter = url to validate
  string url = (string)InValues[0].GetString();    

  var request = HttpWebRequest.Create(url);
  request.Method = "HEAD";
  
  try
  {
    using (var response = request.GetResponse() as HttpWebResponse)
    {
      if (response != null)
      {
        statuscode = Convert.ToInt32(response.StatusCode);
        response.Close();
      }
    }
  }
  catch(WebException ex)
  {
    var response = ex.Response as HttpWebResponse;
    statuscode = Convert.ToInt32(response.StatusCode);
  }

  return statuscode;
}

PowerShell script to validate a URL)
param([string]$url)

Write-Host -NoNewline "Check URL $url ... "

$request = [system.Net.WebRequest]::Create($url)

try
{
    $response = $request.GetResponse()
} 
catch [System.Net.WebException]
{
    $response = $_.Exception.Response
}

Write-Host "done"

# return current http status code
exit [int]$response.StatusCode






File Attachment:

File Name: validate-urls.zip
File Size:3 KB

Best wishes
Robert Stark

Did this answer your question? We would be grateful if you provide a brief comment as feedback. It may also help others who may have encountered a similar problem.
Attachments:
The following user(s) said Thank You: Daniel

Please Log in or Create an account to join the conversation.

Best wishes
Robert Stark

Did this answer your question? We would be grateful if you provide a brief comment as feedback. It may also help others who may have encountered a similar problem.
  • Page:
  • 1

other Languages

de

FlowHeater Home

de en

Imprint/Contact

Privacy Statement

Copyright © 2009-2019 by FlowHeater GmbH.
All rights reserved.

Follow us on

twitter  facebook

YouTube

 de en