Can't access Grafana behind IIS reverse proxy [windows]

Need a lil help!

OS: Windows 10E
webserver: IIS10
Grafana Standalone Windows 4.2
Grafana as Win Service
server IP: 192.168.1.20
installation at: c:\intepub\wwwroot\grafana

ref: http://docs.grafana.org/installation/behind_proxy/

I had my grafana install working fine on the local windows 10 box, but I needed to enable the reverse proxy… So I edited all the settings in both IIS and Grafana.

So the grafana service will start, and it’s listening on the port I configured in my config file - 8200

Now when I try to navigate to the main landing page on the box that’s hosting: xx.xx.xx.xx:PORT/grafana I get a blank black screen in the browser. this happens both on the hosting machine and on a remote client that tries to access via http://HOSTNAME/grafana

Browser inspect console looks like this:

app.js?bust=1493900174975:4 Application boot failed: Error: (SystemJS) XHR error (404 Not Found) loading http://192.168.1.20/grafana/public/app/features/annotations/all.js?bust=1493900174975 Error: XHR error (404 Not Found) loading http://192.168.1.20/grafana/public/app/features/annotations/all.js?bust=1493900174975 Error loading http://192.168.1.20/grafana/public/app/features/annotations/all.js as "./annotations/all" from http://192.168.1.20/grafana/public/app/features/all.js 

Application logs look like this:

t=2017-05-04T05:14:41-0700 lvl=info msg="Request Completed" logger=context userId=3 orgId=1 uname=administrator method=GET path=/ status=200 remote_addr=192.168.1.20:7595 time_ms=23ns size=19227
t=2017-05-04T05:14:42-0700 lvl=info msg="Request Completed" logger=context userId=3 orgId=1 uname=administrator method=GET path=/public/app/features/annotations/all.js status=404 remote_addr=192.168.1.20:7618 time_ms=9ns size=19227
t=2017-05-04T05:16:14-0700 lvl=info msg="Request Completed" logger=context userId=3 orgId=1 uname=administrator method=GET path=/ status=200 remote_addr=192.168.1.20:7618 time_ms=16ns size=19227
t=2017-05-04T05:16:16-0700 lvl=info msg="Request Completed" logger=context userId=3 orgId=1 uname=administrator method=GET path=/public/app/features/annotations/all.js status=404 remote_addr=192.168.1.20:7595 time_ms=13ns size=19227

Config looks like this:

[paths] 
data = /data 
logs = /data/log 
plugins = data/plugins

[server] 
protocol = http
http_addr = 127.0.0.1
http_port = 8200
domain = hostname.com
enforce_domain = false
root_url = http://hostname.com/grafana
static_root_path = c:\inetpub\wwwroot\grafana\public
enable_gzip = false

I have several other applications that are running fine behind the proxy, so I"m not sure what’s up.

Thanks!

What do your outbound rules look like in IIS? Are they rewriting the url the same way as the inbound rules?

@daniellee - Yeah, I think so:

        <rewrite>
            <rules>
                <clear />
                <rule name="ReverseProxyInboundGrafana" stopProcessing="true">
                    <match url="(.*)" />
                    <conditions logicalGrouping="MatchAll" trackAllCaptures="false" />
                    <action type="Rewrite" url="http://192.168.1.20:8200/{R:1}" />
                    <serverVariables>
                        <set name="HTTP_X_ORIGINAL_ACCEPT_ENCODING" value="{HTTP_ACCEPT_ENCODING}" />
                    </serverVariables>
                </rule>
            </rules>
            <outboundRules>
                <remove name="RestoreAcceptEncoding" />
                <rule name="RestoreAcceptEncoding" preCondition="NeedsRestoringAcceptEncoding" enabled="true">
                    <match serverVariable="HTTP_ACCEPT_ENCODING" pattern="^(.*)" />
                    <conditions logicalGrouping="MatchAll" trackAllCaptures="true" />
                    <action type="Rewrite" value="{HTTP_X_ORIGINAL_ACCEPT_ENCODING}" />
                </rule>
                <rule name="ReverseProxyOutboundGrafana" preCondition="ResponseIsHtml1">
                    <match filterByTags="A, Area, Base, Form, Frame, Head, IFrame, Img, Input, Link, Script" pattern="^http(s)?://192.168.1.20:8200/grafana/(.*)" />
                    <action type="Rewrite" value="http{R:1}://seanvree.com/grafana/{R:2}" />
                </rule>
            </outboundRules>
        </rewrite> 

It’s been a while since I did anything with url rewrites in IIS but it looks like maybe outbound relative links are not getting rewritten?

I don’t know. It doesn’t make sense. All my other applications are set up just like this, no issues.

Quick sanity check - how did you install Grafana? From source?

@daniellee -

I used Powershell:

wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-4.2.0.windows-x64.zip 
unzip grafana-4.2.0.windows-x64.zip  

Then I moved the files to c:\intepub\wwwroot\grafana
(.exe sits in \bin)

Then I made a service with nssm.

And launch it via:

C:\inetpub\wwwroot\Grafana\bin\grafana-server.exe --config C:\inetpub\wwwroot\Grafana\conf\custom.ini

Like I said, the program launches fine, and listens on the correct port.

Really appricate the help. I know most people aren’t using IIS.

Grafana is an SPA(Single Page App). The HTML pages are just a starting point before Angular takes over. So the relative outbound urls need to be correct or it won’t work.

I’m not sure if the problem is that the urls to the internal links are incorrect due url rewriting or if the JavaScript is not working for some other reason.

If you manually navigate to the url for the js: http://hostname.com/grafana/public/app/boot.a3e33811.js do you get a 404 or a javascript file?

Googling this: I found someone on Stackoverflow with a similar problem and they linked to this.

@daniellee

Yeah, I know. I’m ALMOST sure it’s a JS problem. Like I said, I have at least 15 other apps that run like this (SPA) behind IIS reverse proxy to internal port, and they all work fine. But, I THINK this might be the only JS program I have.

In answer to your question, this is what I see: (see on left)

Imgur

Yeah, I saw that SO page as well, but I’ve checked all the rules and I CANNOT figure it out. If you see the console errors above, those links are GOOD, so I have no idea why it can’t write them?

Those relative urls look wrong. Googling a bit more:

At a conference today and tomorrow but this is something that is missing from our docs that we would like to fix (we can’t just assume everyone is running nginx or apache). If you don’t get this working, then I will set it up on my Windows machine and then document the url rewrite rules.

@daniellee - I’ll have to take a look at these angular routes. I haven’t done much with this.

Like I said, I REALLY do appreciate your help.

I’m that one jerk who likes to run all the open source stuff on a MS server. HA HA.

I get this with every project I do.

But, I do appreciate it.

I’ll report back when I can take a look at these two sources.

hey @danielle -

So, I looked a bit at this angularJS stuff, and I would agree, that this is probably the ticket.

however,

it’s a BIT over my head. I tried the angular rewrite rules, but I just can’t get it to work.

I’d really like for the DEVs try to replicate this issue.

I don’t think it’s too much to ask to have this running on Microsoft IIS.

LMK.

THANKS

OK, I’ll give it a try and see if I can get it working.

Thanks! No rush buddy. But I’m REALLY interested to see if this works for you “out of the box” If not, and I have to bone up on the angularJS thing, I can do that, but I spent about 2 hours on it last night, and I just didn’t have the knowledge of it to go down that path if it’s not even needed.

LMK.

Hi!

I’m trying to play the same application to run Grafana in a reverse proxy and I get this error

logger=server error=“listen tcp 0.0.0.0:3000: bind: An attempt was made to access a socket in a way forbidden by its access permissions.”

Could you give me a hand?

Thank you!

@ingenieria5 from the docs:

The default Grafana port is 3000, this port requires extra permissions on windows. Edit custom.ini and uncomment the http_port configuration option (; is the comment character in ini files) and change it to something like 8080 or similar. That port should not require extra Windows privileges.