Response Transformer complex jsons

[NF13]

Response Transformer can't remove fields from complex jsons like:

{
	"name": "test",
	"address":
	{
		"fieldtoremove": "test",
		"street": "test 3"
	}
}

I tried the following two operations:

curl -X POST http://kong:8001/apis/cebdca24-91f3-490f-bbaf-d5746c212d52/plugins --data "name=response-transformer" --data "config.remove.json=address.fieldtoremove"

curl -X POST http://kong:8001/apis/cebdca24-91f3-490f-bbaf-d5746c212d52/plugins --data "name=response-transformer" --data "config.remove.json=fieldtoremove"

Is there some other way or is this not working?

Same Issue for Lists like:

[
{
	"fieldtoremove": "test",
	"test": "test"
},
{
	"fieldtoremove": "test2",
	"test": "test2"
}
]

curl -X POST http://kong:8001/apis/cebdca24-91f3-490f-bbaf-d5746c212d52/plugins --data "name=response-transformer" --data "config.remove.json=fieldtoremove"

[thibaultcha]

Hi,

There is no way to do this with the current request-transformer plugin to the best of my knowledge. Contributions welcomed!

[hanselke]

any idea what is lacking in this? dont know lua but a quick search found this https://github.com/harningt/luajson

does it just not have a json library that will work currently?

[jeremyjpj0916]

These methods splitstring and setjsonprop should do the trick when added to the https://github.com/Kong/kong/blob/master/kong/plugins/response-transformer/body_transformer.lua

local json_body_test_one = {data = { id = {"a", "b"},d = "2" }} --decoded json w cjson
local json_body_test_two = {data = { { id = "a", d = "1" }, { id = "b", d = "2" } } }
local config_json_remove = "data.id"

local function dump(o) --Method to print test tables for debugging
   if type(o) == 'table' then
      local s = '{ '
      for k,v in pairs(o) do
         if type(k) ~= 'number' then k = '"'..k..'"' end
         s = s .. '['..k..'] = ' .. dump(v) .. ','
      end
      return s .. '} '
   else
      return tostring(o)
   end
end

local function splitstring(inputstr, sep)
  if sep == nil then
    sep = "%." --Dot notation default
  end
  local t={} ; i=1
  for str in string.gmatch(inputstr, "([^"..sep.."]+)") do
    t[i] = str
    i = i + 1
  end
  return t
end

local function setjsonprop(json_object, path, newvalue)
  local configarray = splitstring(path)
  while (#configarray > 1) do
    json_object = json_object[table.remove(configarray, 1)]
    if(type(json_object) == "table" and #json_object > 0) then
      local recursepath = table.concat(configarray, ".")
        for _, item in pairs(json_object) do
          setjsonprop(item, recursepath, newvalue)
        end
        return
    end
  end
  
  json_object[table.remove(configarray, 1)] = newvalue
end

setjsonprop(json_body_test_one, config_json_remove, nil)
print(dump(json_body_test_one))

Same method can be used to nil and replace elements/lists in nested json (or non nested, won't break old functionality :P ). Would be neat to have someone who has Kong benchmark tests actually run it on some deep nested payloads and see how efficient it is. Once I get motivated enough I might submit a PR, rather someone else do it though hah.

[kris3ci]

FWIW, I have a case where we would like to remove/replace a nested JSON value. Has anyone has a chance to take a look at implementing this improvement?

[jeremyjpj0916]

My code works if you dump into the response transformer or even the request transformer it could too heh. On my growing list of todo's but not a priority atm for me.

[preranashekhar]

I'm wondering if this issue has been fixed in Kong 1.3.0. As it's not working for me, I'm trying to figure out if I'm missing something or that's the way it still behaves.

[Tieske]

@preranashekhar it hasn't changed. But you can try the pre-function or post-function plugin to achieve the transformation you need